Post by James » Sat May 03, 2014 2:59 am

This is either going to be a long, boring post that you don't care too much or understand about - or one of those where you stand up and say "Thank f**k for that, it's about time"....

Going to cover:
  • Jenkins
  • Code sniffer
  • Unit testing
The above is not the total extent of what OpenCart needs (just the basics) but no one disagrees that testing is important, probably one of the most important things! The recent issues with changing the utf8 helper file is just a prime example, if we had extensive / automated testing 99% of these issues would never get into a release.

Jenkins

It's nothing more than a bit of java software designed to run lots of modules to do different little things - or a continuous integration server to the nerd. More info: http://jenkins-ci.org/. Basically add plugins to do your tasks (i.e. 1. Git clone your repo, 2. scan for code syntax errrors, 3. scan for coding standards, 4. Make you a coffee, 5. Email you the results, 6. Deploy ..... i think you get the picture.). You can do most things with Jenkins INCLUDING unit tests, selenium etc etc. Honestly have a play, run it in a VM (we use a 1GB 1 core Hyper-V on our Windows 2012 server and its plenty, but FYI an Amazon Micro instance isn't lol).

Code Sniffer

My favorite test tool for PHP! Even your IDE will support it (NetBeans, PHP Storm and if you really must, Sublime)

Anyone noticed my random commits that have spotted silly coding standard errors? That is code sniffer at work telling us that some code got added that was bad. Here is one:
https://github.com/opencart/opencart/co ... 9120089460

Code sniffer just checks your code to match it against the coding standards of a project, set what ever you want and even create your own rules! A list is here though:
http://pear.php.net/package/PHP_CodeSni ... s/2.0.0a2/

If you have XAMPP installed just use shell to install in about 2 mins.

As a second line of defense our Jenkins server even runs sniffer on the WHOLE OpenCart project on demand.

Maybe soon we will even configure Jenkins to scan through each Pull Request to ensure it passes the standards tests and it can even comment back on GitHub with a pass or fail! All without any admins even looking at the code :)

Unit Testing

To start with, credit goes to: https://github.com/Beyond-IT/opencart-test-suite - seems they started and abandoned the project which is a MASSIVE shame. I sent them an email a while back but no response :( So the solution that has been created has been adapted from an existing source, updated and finished - and importantly it is easy to use.

A new branch has been created on GitHub, forked from the current OC master (please do not submit PR's to it as it is for the test-suite only):
https://github.com/opencart/opencart/tr ... ting-suite

It now has a /test/ folder - this is where anything and everything related to unit testing will live.

If you don't know what unit testing is then you should! In a nut shell it is a way to give your application some thing and expect a specific result in return - i.e. testing it works as it should. Here is an example:

Code: Select all

class CatalogControllerCheckoutCartTest extends OpenCartTest {
	
	/**
	 * @before
	 */
	public function setupTest() {
		$this->cart->clear();
	}
	
	public function testAddProduct() {
		$this->request->post['product_id'] = 28;
		$this->request->post['quantity'] = 1;
		
		$response = json_decode($this->dispatchAction("checkout/cart/add")->getOutput(), true);

		$this->assertTrue(!empty($response['success']) && !empty($response['total']));
		$this->assertEquals(1, preg_match('/HTC Touch HD/', $response['success']));
		$this->assertEquals(1, preg_match('/119\.50/', $response['total']));
	}
}
Pretty easy to understand eh? So why use it?

Imagine you change something, a method, controller, model whatever - or you even add a new module - the idea is that you can run ONE command and it will report syntax, unit test fails, coding violations (and even what file too)

The good news is that we're already doing the unit tests - and hopefully people will read the readme (yes there is one and will be expanded even more soon!!)...then start using the testing tool.

As always, love to get feedback!

Anyone with some knowledge of the above with some additions?

But a final word...yes I will also look into Selenium and Vagrant....it would be pretty cool to run Jenkins, do all the code tests, boot up a VM, install OpenCart, run Selenium tests - then send us a pretty email (never know might even make it a responsive email lol), but we are a long way off that just yet.

J
Last edited by i2Paq on Wed May 07, 2014 3:23 pm, edited 1 time in total.
Reason: Moved + sticky

User avatar
Active Member

Posts

Joined
Wed May 27, 2009 6:07 am
Location - Leeds, UK

Post by rph » Sat May 03, 2014 3:48 am

Great work!

On the coding syntax I noticed a missing space in leading braces got missed:

Code: Select all

}else { 
and a few "else if"s were still mixed in with the "elseif"s. I didn't comment since it's such minor stuff but if the cleanup is being automated it would be cool to add a rule. There are also some issues with rogue whitespace (spaces mixed in with tabs, useless whitespace at the end of a line) and the mix of Windows and Unix CRLFs in files. It's probably too disruptive to GitHub tracking to make those changes in the middle of the development cycle but it would be worth doing at the very end or before work starts on 2.1.

I'd be interested to hear what others think about unit testing OpenCart. I like the idea but I don't think it'll be useful until there's a radical rewrite of the controllers and some libraries. The methods are just too large and do too much. In areas like the cart class you have nesting that goes nine levels deep. That's a truckload of cyclomatic complexity and writing the tests and debugging the results is going to be tough.

-Ryan


rph
Expert Member

Posts

Joined
Fri Jan 08, 2010 5:05 am
Location - Lincoln, Nebraska

Post by villagedefrance » Sun May 04, 2014 4:58 pm

Fantastic news James !

As you know I have been doing a lot of digging in the code recently and like rph mentioned above, it is not pretty in places. We really need to do something about this, so Testing tools are great news!

Just in case, there is also Qunit, the jQuery testing tool. Could be useful.

If testers are required, let me know.

OpenCart custom solutions @ https://villagedefrance.net


User avatar
Active Member

Posts

Joined
Wed Oct 13, 2010 10:35 pm
Location - UK

Post by James » Sun May 04, 2014 7:04 pm

Thanks guys, it's taking its time but all these little changes are certainly helping! I do think the Jenkins part is great though - if you want access to the reports you can register on our Jenkins box here: http://109.239.111.4:8080/signup - i will assign permissions if you do.

You can also see any current violations. I will also publish the base config files so if there is any real "Jenkins Pro's" out there who can offer something more, awesome. I will put these into the /tests/ folder in the test suite as that is where they really should live so anyone can use them (and contribute etc)

@rph where did you spot that }else { - I see one inside a 3rd party plugin but don't want to touch them, only OC code.

I'll checkout the SQL now too, not sure why they are latin..probably just a setting from our local dev machines.

Is Qunit not similar to Selenium? I'll certainly take a look though, but TESTERS are always needed :)

Wish we could have a competition style leader board, who finds the most bugs/code violations/suggestions etc - maybe even a $50 Amazon giftcard each month - or just something to say thanks!! Anyone know some cool 3rd party app for it?

J

User avatar
Active Member

Posts

Joined
Wed May 27, 2009 6:07 am
Location - Leeds, UK

Post by James » Mon May 05, 2014 1:05 am

Haha, that was just just a tease! Although I'm seeing Daniel in a few weeks, i'll get him drunk and try to get a few Amazon voucher values haha..If there was such a system/tool it would be a great community addition.

TBH i was trying to ignore 1.5.6.x - it is almost end of life with 2.0 soon so my efforts are focused on that for these perfections, even OpenBay is getting an overhaul and update to ensure it is as good standards wise.

Not to mention the sheer volume / value of modules for 1.5.6.x - any small code change could render way too many modules requiring an update...i.e.

Think a vqmod search for:
$productIdArray
And we change to (the correct way)
$product_id_array

* yeah the standards are wrong but right now it just works.

It could have serious effects on many modules, less sales, less revenue etc - even the last few releases seem to have taken a toll with some devs.

2.0 is looking pretty fresh now, just these dam space/tab issues. Once sorted I will add in even more rules though :)

But, if you find more, a PR will always be appreciated! just put @jamesallsup so I pick it up and can also re-run the tests too.

J

User avatar
Active Member

Posts

Joined
Wed May 27, 2009 6:07 am
Location - Leeds, UK

Post by rph » Mon May 05, 2014 12:50 pm

James wrote:TBH i was trying to ignore 1.5.6.x - it is almost end of life with 2.0 soon so my efforts are focused on that for these perfections, even OpenBay is getting an overhaul and update to ensure it is as good standards wise.
So is 1.5.6 officially the last in the 1.5.x line?
It could have serious effects on many modules, less sales, less revenue etc - even the last few releases seem to have taken a toll with some devs.
Yeah, I haven't sold anything in a week. I don't know if that normal season up and down or a result of all the changes lately. It certainly is discouraging.
2.0 is looking pretty fresh now, just these dam space/tab issues. Once sorted I will add in even more rules though :)

But, if you find more, a PR will always be appreciated! just put @jamesallsup so I pick it up and can also re-run the tests too.
It's definitely a big job. I did a quick check on 2.0 and there are 1300+ hits on 400+ PHP files just for trailing whitespace.

-Ryan


rph
Expert Member

Posts

Joined
Fri Jan 08, 2010 5:05 am
Location - Lincoln, Nebraska

Post by i2Paq » Mon May 05, 2014 1:16 pm

James wrote:This is either going to be a long, boring post ....
boImageooring
James wrote:that you don't care too much or understand about - or one of those where you stand up and say "Thank f**k for that, it's about time"....
Sorry, couldn't resit ;D

I'm not a coder, so to me it is a kind of hokes-spocus, but I understand that these are pieces of software that you let the code run thru to see if it is according to standards?

We have these tools to, we use them before compiling our software.

Great step forward!

f*cking awesome :D

Norman in 't Veldt
Moderator OpenCart Forums

_________________ READ and Search BEFORE POSTING _________________

Our FREE search: Find your answer FAST!.

[How to] BTW + Verzend + betaal setup.


User avatar
Global Moderator

Posts

Joined
Mon Nov 09, 2009 7:00 pm
Location - Winkel - The Netherlands

Post by James » Mon May 05, 2014 7:18 pm

WTF guy, how many quotes lol
So is 1.5.6 officially the last in the 1.5.x line?
That's not official, but I am supporting it for bugs as long as possible, features will be little if none at all now. 2.0 is way to close to be releasing more older versions (unless we find other reasons to do so :) ). Personally we did several new gateways added to 2.0 that did not get into 1.5.6
Yeah, I haven't sold anything in a week. I don't know if that normal season up and down or a result of all the changes lately. It certainly is discouraging.
Yeah a few reported the same too, but we didn't notice much change....hence less releases (but still quite frequent if possible!). The global sales stats are on par though.
It's definitely a big job. I did a quick check on 2.0 and there are 1300+ hits on 400+ PHP files just for trailing whitespace.
Yup - but my IDE (PHP Storm is awesome, just open the files and save - gets rid of that crap!) Latest report below - A day at the most.

Image
Sorry, couldn't resit ;D
Haha, I did say it was boring :) BUT if it makes a more solid test case for OC then who cares right?

And one bit of final good news, the chaps who wrote the original test suite got in touch, they use OC still and have even more ideas so looks like we are onto a winner here :)

User avatar
Active Member

Posts

Joined
Wed May 27, 2009 6:07 am
Location - Leeds, UK

Post by rph » Tue May 06, 2014 3:33 am

There's been an explosion in knockoffs since sales figures got published. Maybe it's that.

-Ryan


rph
Expert Member

Posts

Joined
Fri Jan 08, 2010 5:05 am
Location - Lincoln, Nebraska

Post by James » Tue May 06, 2014 4:28 am

Where did they get published mate? - I am guessing the actual module pages?

I'm not trying to cover up / hide / be sneaky about anything but financials, sales volumes and sensitive info should always be on a need to know. I don't see the point in having the volume of sales for a module on the page unless it is for free (or at least give the option to the seller maybe?)

RPH if your right and there was an increase in piracy since the stats went public then we need to do something.

But, lets split this to another thread....can a mod do that please (for some stupid reason an Admin can't do that!!)

User avatar
Active Member

Posts

Joined
Wed May 27, 2009 6:07 am
Location - Leeds, UK

Post by rph » Tue May 06, 2014 12:17 pm

Sorry, didn't mean to imply they were pirated. They're just modules that do the same thing and often have the same or very similar name.

-Ryan


rph
Expert Member

Posts

Joined
Fri Jan 08, 2010 5:05 am
Location - Lincoln, Nebraska

Post by James » Tue May 06, 2014 3:05 pm

rph wrote:Sorry, didn't mean to imply they were pirated. They're just modules that do the same thing and often have the same or very similar name.
Ahhh, ok that is less worrying. Still I lose track of the number of a "certain countrys" websites that post mods in the forums.

User avatar
Active Member

Posts

Joined
Wed May 27, 2009 6:07 am
Location - Leeds, UK

Post by openmandrake » Wed May 07, 2014 4:02 am

Hi,

thx for inviting me to join the development here.

I started the Unit Testing project about a year ago and could cover all my usecases. I abandoned it, because we didn't need more functionality and never expected any interest... All in all it is great that there is an effort to introduce automated testing to the OpenCart project. In my opinion this would benefit the project a lot!

There is a lot which is possible in testing these days. Probably not everything makes sense at this stage. In the following I mention a couple of my toughts:
  • CI Server: There are many of them (Jenkins, CruiseControl, Continuum, ...). In OSS project Travis CI seems to gain high popularity. For a couple of reasons: free hosting for OSS Projects, great out of the box support for many programming environments, very easy to get started, etc. So in my opinion it would be best to use Travis CI. It also has nice badges which can be integrated on webpages like github :)
  • Unit Testing: There is already a great Test Base in Place now! Great! The only thing which is important, in my opinion, is a set of conventions for proper naming and organizing the tests. Otherwise, it could easily happen (when more people work on it), that it gets a mess fast. Kent Beck (the inventor of unit testing) has some good advice: http://stackoverflow.com/a/155475/1574174
  • Acceptance Testing: For this we would have to write down a thorough and concise list of prioritized userstories (this should in the best case cover all processes/task a user can do with opencart) and write a simulation for that. A userstory would be like: "An anonymous user wants to create an account. After that she is logged in and an email was sent to her". Then we would need to write a simulation for each userstory and test all the criterias afterwards (similar to unit tests). Sauce Labs offers an amazing environment for running the tests (https://saucelabs.com). It offers a free version for OSS projects and integrates nicely into Travis.
I could definitely assist in these steps, is there any general plan or something? My interest would be acceptance testing, however as this is a huge task, i am quite sure I cannot do this alone...

Checkout our extensions, or our open source projects


New member

Posts

Joined
Wed Oct 31, 2012 3:05 am

Post by i2Paq » Wed May 07, 2014 3:24 pm

I moved it (as requested by James) to the Release Candidate forums so more people can contribute to the discussion.

Norman in 't Veldt
Moderator OpenCart Forums

_________________ READ and Search BEFORE POSTING _________________

Our FREE search: Find your answer FAST!.

[How to] BTW + Verzend + betaal setup.


User avatar
Global Moderator

Posts

Joined
Mon Nov 09, 2009 7:00 pm
Location - Winkel - The Netherlands

Post by James » Wed May 07, 2014 4:51 pm

Thanks i2Paq...

Welcome Stefan (apologies I didnt get back to you on Skype I have been a little busy!) - as a back storey we managed to get in touch eventually and was invited over to help with our new testing systems for OC.

So for now we are rolling with Jenkins, easy to use, configure etc and seems to have a module for anything you need.

Unit tests are being written right now but hopefully people will start to submit some more, they are just organised how the normal OC file structure is (makes sense and you always know where to get something).....but there is A LOT of them to do.

Your final point about user stories is key, selenium/SL can be used for most of these tests but we will also need a certain amount of man power ( I am thinking Odesk and a private bug tracker).

Anyway drop me a line when your on SKype later and I can introduce you to my guys heer and share some thoughts on moving forward!

J

User avatar
Active Member

Posts

Joined
Wed May 27, 2009 6:07 am
Location - Leeds, UK

Post by andrei.bogdan » Wed Jun 13, 2018 5:08 pm

Hi guys,

I took the time to read your posts and I am very happy to hear that testing has been adopted for the project! I will try to get involved as much as I can.

I also cloned the repo on my local machine and started checking the files from 'phpunit' folder. To my surprize, I found actual db queries in those tests, which I know is not a good practice for unit testing, as stated here: https://phpunit.de/manual/6.5/en/database.html > Difficulties in Database Testing

1. As far as I know, we should mock objects in order to test code that uses the database, as explained here: https://codeutopia.net/blog/2009/06/26/ ... -database/

2. If we really need to test against actual db results, maybe we can include this in the Acceptance section (with tools such as Selenium).

If I am wrong then please excuse me, if not I can say that we are totally going in the wrong way about creating actual db queries in our tests.

Best regards,

Andrei
andrei.opencart[at]yahoo.com
Themes and Extensions


Active Member

Posts

Joined
Mon Mar 25, 2013 4:52 am

Post by schiggi » Tue Jul 03, 2018 6:41 pm

Would be great to revive the testing as it not only helps the development of the core product but also the extension developers to maintain a cleaner code and check, if their extensions don't break any core functions.

New member

Posts

Joined
Tue May 13, 2014 4:23 am

Post by TiredFingers » Tue Nov 27, 2018 8:50 pm

Ok guys, it's great. But can I use it today to make tests?

Newbie

Posts

Joined
Sun Sep 04, 2016 3:48 am

Post by joeymagsino » Sun Mar 31, 2019 4:52 pm

thanks guys i'm working on a project and I've find the test suite useful. I'm looking forward for more improvements.

Newbie

Posts

Joined
Thu Apr 05, 2012 1:21 pm
Who is online

Users browsing this forum: No registered users and 12 guests