Run module tests in OXID eShop 6

As follow up for Run tests for OXID eShop 6 here’s how to get module tests running for OXID eShop 6.

Let’s assume we have a metadata 2.0 module (own module namespace, installable with composer) in some public github repository (original or your own fork). Let’s use the OXIDProjects oxid-module-internals as example.

In case you just want to install the module and be good, check module’s README.md or documentation about how to install via composer. Get your OXID VM up and do vagrant ssh.

Example:

cd /var/www/oxideshop
composer require oxid-community/moduleinternals

The module’s source code is downloaded into the vendor directory (vendor/oxid-community/moduleinternals).
As the shop still expects module source code to be located inside source/modules, the module’s composer.json contains an entry for the ‘target-directory’ in the extra section and the OXID composer plugin takes care all necessary files are copied into that target-directory.

"extra": {
    "oxideshop": {
      "target-directory": "oxcom/moduleinternals"
    }
},

Best way if you want to actually develop the module is to directly git clone the module
into source/modules/target-directory.

cd /var/www/oxideshop/source/modules
git clone https://github.com/OXIDprojects/oxid-module-internals.git oxcom/moduleinternals

As you did not use the composer installation way, you need now to manually register the module’s namespace. Add the module namespace to the shop’s main composer.json, either into autoload or autoload-dev section. Make sure it points to the correct directory.

"autoload": {
    "psr-4": {
      ...
      "OxidCommunity\\ModuleInternals\\": "./source/modules/oxcom/moduleinternals"
    }
  },

Then do

cd /var/www/oxideshop
composer dump-autoload

Disregarding of you way of installation, doublecheck you have the module tests located
in source/modules/target-directory/Tests.

Edit the shop’s test_config.yml

 ...
 partial_module_paths: oxcom/moduleinternals
 ...
 run_tests_for_shop: false
 run_tests_for_modules: true
 ...

and run

cd /var/www/oxideshop
vendor/bin/reset-shop
vendor/bin/runtests
Posted in howto, oxshop, oxshop6 | Tagged , , , | Leave a comment

Run tests for OXID eShop 6

Nice thing about OXID eShop, they provide a development environment. Without big effort you can get have the shop up and running on a virtual machine. The current blog post will give some hints for how to run the shop tests that come with the shop. We will cover how to run module tests in a follow up post.

NOTE: my host is a Macbook Pro with macOS High Sierra 10.13.2.

Get the vm up and running

First have a look at https://github.com/OXID-eSales/oxvm_eshop/tree/b-6.0.
Get vagrant (Vagrant 2.0.1 was used in my case) and all that stuff installed on your machine
as described in section Dependencies.
Then do

git clone -b b-6.0 --recursive https://github.com/OXID-eSales/oxvm_eshop.git

Inside directory oxvm_eshop create a file personal.yml with the following content:

---
php:
  version: 5.6
  composer:
    github_token: 
selenium:
  install: true
vagrant_local:
  vm:
    name: oxideshop_201712_PHP56

The github token prevents you from running into trouble with rate limits. Selenium is needed to run acceptance tests and PHP version can be adjusted as you need it. I prefer to have multiple virtual machines so it’s better to name them individually. Switching PHP versions in one vm is PITA, much easier to create different ones for each PHP version you need and switch between VMs.

Now run vagrant up to create and provision the virtual machine. Once you got the virtual machine up and running, there are two ways to get the shop installed.

NOTE: the OXID VM has a shared folder oxvm_eshop/oxideshop where the shop code needs to be installed. This folder points to /var/www/oxideshop/ inside the vm. The shop can be accessed at http://www.oxideshop.dev once it is installed.
You can try to install the shop in another directory but the paths might be hardcoded by accident on some tests so if anything weird happens better stick to the defaults.

Install the shop, git clone way

When you want to develop something for the shop core, this is the best way to go. In case you are
not a contributor to OXID eShop CE repository you will have to use a fork and send in changes to shop core via pull requests.

Inside oxvm_eshop directory on your host call vagrant ssh to connect to VM console.
You end up in /home/vagrant, so first change to /var/www directory. Then git clone (oxid repo or your fork) directly into /var/www/oxideshop in vm, in host you find the files in oxvm_eshop/oxideshop directory.

git clone https://github.com/OXID-eSales/oxideshop_ce.git oxideshop 

You can switch the branch/tag as you like if you do not want the master but a specific release.

Then you need to run composer install, so run

cd oxideshop
composer install

When this is done (takes a while on first run) you get asked in the end about some paths, as we do the default installation no need to enter anything.

...
Creating the "test_config.yml" file
Some parameters are missing. Please provide them.
shop_path (source):                
shop_tests_path (tests): 
partial_module_paths (null): 

What we no still need is the infamous source/config.inc.php file, so

cd source (cd /var/www/oxideshop/source)
cp config.inc.php.dist  config.inc.php

And change the placeholders in config.inc.php to

$this->dbType = 'pdo_mysql';
$this->dbHost = '127.0.0.1'; // database host name
$this->dbPort  = 3306; // tcp port to which the database is bound
$this->dbName = 'oxid'; // database name
$this->dbUser = 'oxid'; // database user name
$this->dbPwd  = 'oxid'; // database user password
$this->sShopURL     = 'http://www.oxideshop.dev'; // eShop base url, required
$this->sSSLShopURL  = null;            // eShop SSL url, optional
$this->sAdminSSLURL = null;            // eShop Admin SSL url, optional
$this->sShopDir     = '/var/www/oxideshop/source';
$this->sCompileDir  = '/var/www/oxideshop/source/tmp';

Alternately you could run the web setup, but this will delete the Setup directory if you are
not careful.

Check if you installation is ok:

cd /var/www/oxideshop
vendor/bin/reset-shop

No output is a good sign here ;). Open a browser, open http://www.oxideshop.dev and you should see the shop start page.

Install the shop, compilation way

This is the way to would install a ‘real life’ shop. Structure of shop installation is different than the ‘git clone’ way because now you have all the ship sources inside the vendor folder
and only necessary stuff is copied to source directory.

vagrant ssh 
composer create-project oxid-esales/oxideshop-project /var/www/oxideshop dev-b-6.0-ce

Again at the end of installation you get asked for

Creating the "test_config.yml" file
Some parameters are missing. Please provide them.
shop_path (source):                
shop_tests_path (tests): /var/www/oxideshop/vendor/oxid-esales/oxideshop-ce/tests
partial_module_paths (null): 

The composer project does create a source/config.inc.php file but you still need to either run shop web setup or modify that file manually. Enter the values same as above. Now also edit the test_config.yml file, you need to explicitly enter the shop_setup_path:

shop_setup_path: /var/www/oxideshop/vendor/oxid-esales/oxideshop-ce

Run shop tests

When you run acceptance tests it’s nice to see what actually is going on. The OXID vm is headless, but you can use VNC viewer for remote desktop.

There are some scripts in vendor/bin directory that are used for running tests:

  • vendor/bin/reset-shop resets the shop to default database and demodata (admin/admin). In case you did add your own test data, dump the database before reset.
  • vendor/bin/runtests runs the tests as defined in test_config.yml.
    NOTE: before tests start, shop unified namespace classes are regenerated so it might take some time before the actual output starts. If the script gives no output at all you should check the test_config.yml for misconfiguration (no tests available) and source/log/EXCEPTION_LOG.txt.
  • vendor/bin/runtests-selenium runs the acceptance tests as defined in test_config.yml.
    NOTE: atm shop acceptance tests run with selenium and still need azure theme.

Call vendor/bin/reset-shop followed by vendor/bin/runtests. All tests (more than 8000 for CE) should pass locally.

Troubleshooting composer
In case you changed something in composer.json and want a clean state or composer gets a hiccup, try to do

composer clear-cache
rm composer.lock
rm -r vendor
Posted in howto, oxshop, oxshop6, virtual machine | Tagged , , | Leave a comment

Shop in the box

My last post described the first part of my new development environment, the vagrant basebox. The box will be provisioned with puppet, a collection of manifests can be found at github. In the ARCHIVE directory of this repository there’s a collection of VM example configurations.The manifests contain all I currently need for my pet projects and the collection will grow and change as time progresses.

The vagrant VM as today uses

  • Vagrant 1.7.2
  • Puppet 3.7.3
  • Hiera, a key/value lookup tool for configuration data. Hiera is included in Puppet 3.
  • VirtualBox 4.3.20
  • The base box is a slightly modified version of hashicorp/precise32 (Ubuntu 12.04 LTS), for more information click here.
  • vagrant-vbguest (0.10.0)
  • vagrant-hostmanager (1.5.0)

Projects are organized as follows:
Each project has one (or more) directories with the projectcode and a directory vmconfig containing the virtual machine configuration.

vmconfig explained

The Vagrantfile describes how to configure and provision the virtual machine for the current project. Directory .vagrant is created by Vagrant. The hiera.yaml file is to be found in a directory named config. Lookup hierarchy is common (e.g. spanning several projects) -> project specific -> private (e.g. developer’s own) for development and production, related yaml files are located in directory environments. Also in directory environments lives a file named site.pp specifing which puppet modules are needed for the current project vm. As I keep the puppet modules in a github repository separated from possible projects, the puppet modules directory inside project’s vmconfig directory is always a link to the actual puppet modules directory.

Getting a project on a virtual machine up and running is dead easy (well, ok, at least locally and for me it works just fine 😉 ).

For convenience and as I have plans to play around with it soonish, let’s install an OXID eShop Community Edition (CE) as project. The puppet manifest named oxideshopce downloads the newest official CE relase package from OXID and unzips it into my chosen target directory. I’ve no script for an automatic shop setup yet, so after the vm is up and running, you still need to manually click through the shop setup steps at the moment.

Six steps to get the project prepared:

  1. Create a directory MYPROJECT.
  2. Inside MYPROJECT directory create a directory theproject. This folder will be synched to the vm.
  3. Inside MYPROJECT directory do git clone https://github.com/hkreuter/deve2015.git
  4. Copy the oxshop stuff from deve2015/ARCHIVE to MYPROJECT/theproject/vmconfig.
  5. Inside MYPROJECT/theproject/vmconfig create a link to deve2015/modules: ln -s ../../deve2015/modules/ modules
  6. Inside directory MYPROJECT/vmconfig run vagrant up

the project

If all works as desired, the virtual machine is ready and provisioned after the first vagrant/puppet run.

the shopThe Vagrant hostmanager (see new vagrant basebox post) takes care of updating the host machine’s /etc/hosts file. After the box is up, the shop can be accessed at http://oxpl, the phpMyAdmin http://oxpl/phpmyadmin (login/pwd is root/root).

Posted in howto, oxshop, tools | Tagged , , , | Leave a comment

New vagrant basebox

New year, new development environment 😉

  • I want to have an environment that can be set up with “one click”.
  • I want to have an environment that is reusable for different projects
    and can be customized to fit different project needs.

What I’ll do is use vagrant with puppet to create the development environment and use hiera to make it easily customizable.

There’s some old post of mine where I started using vagrant with chef. That was some time ago and it looks like puppet is the one to chose over chef nowadays. The earlier decision to use chef was probably due to being hungry and liking the cookbook idea too much.

I also created my own basebox then, which is nice to know how it’s done but for usual purposes the standard Ubuntu 12.04 LTS box works just fine. Almost at least, I want to use puppet 3 on the basebox, so I’ll use a slightly altered version of hashicorp/precise32.

Caveat: vagrant new versions come up quite regularly, so you should check for new vagrant versions on a regular basis to keep the environment up to date. Current vagrant version is 1.7.2.

Download the base box:
vagrant box add hashicorp/precise32
This is a simple 64-bit Ubuntu 12.04 LTS box that has Chef/Puppet pre-installed.

Run vagrant init, adapt the Vagrant file to your needs, e.g. change the vm name. You’ll find the Vagrantfile I used at github.com. Do vagrant up, vagrant ssh and install whatever you thing you need on your basebox.

For example, to enable the repository for Ubuntu 12.04 Precise Pangolin:

  • wget http://apt.puppetlabs.com/puppetlabs-release-precise.deb
  • sudo dpkg -i puppetlabs-release-precise.deb
  • sudo apt-get update
  • sudo apt-get install puppet

NOTE: virtual machine will need a reboot to use the newly installed puppet version.

Make sure the virtualbox guest additions are installed on that box as well. You can either use a vagrant plugin named vagrant-vbguest that takes care of the guest additons for you
vagrant plugin install vagrant-vbguest or manually install the guest additions:

  • sudo apt-get install dkms
  • sudo apt-get install virtualbox virtualbox-qt virtualbox-dkms virtualbox-guest-dkms
  • sudo apt-get installlinux-headers-3.2.0-23-generic-pae
  • wget http://download.virtualbox.org/virtualbox/4.3.8/VBoxGuestAdditions_4.3.8.iso
  • mkdir ~/cdrom
  • cd cdrom
  • sudo ./VBoxLinuxAdditions.run

Either way, the vm now is larger as it needs be, no matter if you manually install the guest additions on the vm or use the vagrant plugin. The disk resizes when adding the guest additions but does not shrink automatically when deleting the guest additions iso image. I could try to resize it using zerofree but there’s other interesting stuff to be done.

NOTE: history -c erases the bash history, might come in handy before repackaging the box.

Repackage the box:
vagrant halt
vagrant package --base BOX_NAME --output YOUR_BASEBOX_NAME.box
and add it to your boxes:
vagrant box add YOUR_BASEBOX_NAME.box --name YOUR_BOX_NAME

Now we’re ready to start provisioning the vm, which will be the topic of the upcoming next post.

Posted in howto, tools, virtual machine | Tagged , , , | Leave a comment

Out of the box…

… or how to set up your development environment with Vagrant.

I use a Macbook Pro, so first step is to download the newest Vagrant.dmg and install it on the Mac.

We need a virtual box vm that later will be packaged as base box. See here for more info. There are VirtualBox vms around that can be used as base boxes for vagrant, but it’s more fun to make one.

A “box” is the base image used to create a virtual environment with Vagrant. It is meant to be a portable file which can be used by others on any platform that Vagrant runs in order to bring up a running virtual environment.

A command line only vm is enough. Usually the base box is later downloaded from somewhere, so better keep it small. Download ubuntu-11.10-alternate-i386.iso and set up a VirtualBox vm:

  • 8GB VMDK disk
  • 512 MB Ram
  • port forwarding from host port 2222 to vm port 22
  • add a cdrom for ubuntu-11.10-alternate-i386.iso

Start the VM, when the installation screen pops up, press f4 and select command line only installation.
Also see http://wiki.ubuntuusers.de/Alternate_Installation.

Follow the installation steps. The vm gets the name vagrant-ubuntu, user name/pwd is vagrant. For now we stick partly to the conventions, but in principle you can chose whatever names you like.
After installation is complete, log in to console and call su to become root. In case you get an authentication error, try to do ‘sudo passwd root’ first.

Call ‘visudo’, set the admin group to no password (add/modify %admin ALL=NOPASSWD: ALL), also add a line ‘Defaults env_keep = “SSH_AUTH_SOCK”‘.

Add an admin group in case it does not yet exist (addgroup admin) and add our user vagrant to this group (usermod -G admin -a vagrant). Restart sudo with ‘/etc/init.d/sudo restart’. Exit out of the root account and call ‘sudo which sudo’. This should not ask for a password and simply spit out something like /usr/bin/sudo.

Now install (as sudo user):

  • ruby
  • rubygems
  • chef (do ‘gem install chef’)
  • ssh

Finally install the guest additions for virtual box. Do

  • sudo mount /dev/sr0 /media/cdrom
  • sudo /media/cdrom/VBoxLinuxAdditions.run

It will complain that the vm has no graphical interface installed but according to friend google you can ignore the complaints. And voila, with ssh -p2222 vagrant@localhost we can log in to the vm.

One hint found: set UseDNS to no in /etc/ssh/sshd_config for speeding up ssh when the host is not connected to the internet.

To be able to log in to the vm without a password, we need to configure ssh authentication with a public key.
On the Mac run

  • ‘ssh-keygen -t rsa -f vagrant’ to generate a key pair named vagrant
  • cat ~/.ssh/vagrant.pub | ssh -p2222 vagrant@localhost ‘umask 077; cat >>.ssh/authorized_keys

Calling ‘ssh -p2222 -i ~/.ssh/vagrant vagrant@localhost’ works, no password requested.

Now it’s time to box the vm.

On the host create a directory named Vagrant and inside it one called vagrant_box.
Change into vagrant_box and do

  • vagrant init
  • vagrant package –base vagrant-ubuntu-1110 –output Dev2013Master.box

to package the box. Dev2013Master.box is less than 500 MB in size.

Time to set up a project. In folder Vagrant create another subfolder named vagrant_test and change into it. Inside add another folder ‘myproject’ and in there some file dummy.txt. Do

  • vagrant init
  • vagrant box add dev2013test ../vagrant_box/Dev2013Master.box

to create the Vagrantfile and import the box.

Add the following lines to the Vagrantfile.

  • config.vm.box = ‘dev2013test’
  • config.ssh.private_key_path = “~/.ssh/vagrant”
  • config.ssh.username = “vagrant”
  • config.vm.forward_port 80, 8080
  • config.vm.share_folder “myproject”, “/myproject”, “./myproject”

The default shared folder is /vagrant in the vm, we add another one that will be found in /myproject on the vm.

Now we only need to call ‘vagrant up’ and can log into the vm with ‘vagrant ssh’. In /myproject on the vm we find the expected dummy.txt file.

To destroy the complete vm (dev2013test) call ‘vargant destroy’. ‘vagrant reload’ restarts the vm after e.g. changing the Vagrantfile. ‘vargant suspend’ suspends the vm, ‘vagrant resume’ starts it again. For more commands call ‘vagrant -h’.

… to be continued.
Next step will be to get a web project up and running.

Posted in howto, virtual machine | Tagged , , | Leave a comment