HOWTO: Have a test & live environment with source control

Hello.

WHAT?
This is an informational post which documents the steps *I* took (many ways to kill a cat) to put my local installation in source control and then establish a workflow of:
--> do a change on my local website (ie on my PC)
--> test it via http://localhost/mywebsite
--> put the code into source control making a comment of what i did (I can also reverse botched changes)
--> ftp'ing changes to my live site on my WebHost (upload a database dump if necessary)

BUT WHY?
Initially I thought I would only need a live website doing all changes directly. Then I discovered that not everything can be customised through the Admin panel, and that I needed to do more complicated code type changes to OpenCart files. This made me realise I needed a test website on my local machine in addition to my live website meaning a could try changes out rather than messing up my live website. Of course very often I need to reverse a lot of the changes I tried on my localmachine, also, it was difficult for me to keep a track of exactly what I had changed and why. Welcome to source control.

HOW?
0. If on Windows:
-------------------------------------------------------------------------------------------------
I assume you have WAMP installed and have downloaded the latest version of OpenCart

Get Cygwin installed
* Un-install Git (if it is already installed)
* Make sure that your PATH environment variable no long has anything git related in it
* Install Cygwin, select "git: Distributed version control" in the Devel package, install
* What's great about Cygwan is that you CAN paste commands into it

Get Git configured
* Open Cygwin
* Run: git config --global user.name "Joe Doe"
* Run: git config --global user.email johndoe@example.com
* Run: git config --global color.ui true
* Run: git config --list

1. GET FILES INTO PLACE & SOURCE CONTROL
-------------------------------------------------------------------------------------------------
Get files into place:
* Open Cygwin terminal and go to C:\wamp\www\, run: cd C:/wamp/www (stay there!)
* mkdir mysite
* Copy files in opencart-1.5.6.4\upload\*.* into <mysite> (I used Windows Explorer)
* Copy (and replace) files in themefolder (eg theme439(full)\*.* ) into <mysite>

I like unix line endings so make everything unix line endings
* First ensure you have dos2unix v6.0 or above, run: dos2unix --version
* ONLY if you have v6.0 or above, run: find ./ -type f -exec dos2unix {} \;

Ignore the right stuff and do initial commit:
* In <mysite> create .gitignore (paste in contents from .gitignore attached)
* git won't commit empty folders, so run: find . -type d -empty -exec touch {}/.gitkeep \;
* See which folders were empty, run: find ./ -name .gitkeep
* git init; git add .; git commit -m "Opencart 1.5.6.4, TemplateMonster template #50756"
* Check that you are ignore the correct files, run: git clean -ndX
* You may as well delete any Thumbs.db, run: find . -type f -name "Thumbs.db" -exec rm -f {} \;
* To show since Thumbs.db were ignored, deleting them doesn't change status, run: git status

Switch Magic Quotes OFF
* Edit .htaccess.txt and ensure this is UNcommented (line 39): php_flag magic_quotes_gpc Off
* git add .; git commit -m "Switch Magic Quotes OFF"

Suppress SQL Warnings:
* Edit \system\database\mysql.php
Add this line:error_reporting(E_ALL ^ E_DEPRECATED); //Michelle suppress warnings for now
Where:Directly under "<?php:"
* git add .; git commit -m "Suppress mysql warnings (for now)"


2. CREATE LOCAL DATABASE & PRIVILEGED USER (both having the same name)
-------------------------------------------------------------------------------------------------
* Go to http://localhost/phpmyadmin > Users > Add User
* Username=somename(or whatever you want), Host=Local, password=yourpassword
* Select "Create database with same name and grant all privileges" (only)
* Click Go
* Check that a database was also created: click Databases

3. RUN OPENCART INSTALLATION LOCALLY
-------------------------------------------------------------------------------------------------
* Go to http://localhost/<mysite>/
* For (1), leave all defaults as is, enter in DB connection credentials
* For (2), I set the OpenCart administration credentials to what I set for the database
* DO NOT YET CLICK on your "OnlineShop" or "Administration"
* Go take a look at config.php and /admin/config.php
(git status returns nothing BECAUSE these two files are in .gitignore)
* Put a copy of localhost config files into sourcecontrol for safe keeping, run:
cp config.php config.php.localhost
cp admin/config.php admin/config.php.localhost
git add .; git commit -m "config.php files for localhost"
* Now to see files that get generated as you're using Online Shop, go http://localhost/mysite/
* Then run: git status AND run: git clean -ndX (it will show an increasing list of ignored files)
* Repeat with http://localhost/mysite/admin/

4. GET YOUR LATEST COMMITTED VERSION RUNNING ON YOUR WEBHOST
-------------------------------------------------------------------------------------------------
* You need only the files specifically checked into git to be uploaded to your Web Host, so run:
* git archive HEAD --format=zip > ftpTheseFilesToWebhost.zip
* Now use an ftp tool like filezilla to ftp the unzipped archive to your Webhost
* On your webhost, make a copy of .htaccess.txt but remove the ".txt" extension (so it will be used)
* Create a new mysql database on your Webhost
* Re-run the OpenCart installation via http://mydomainname or http://mydomainname/mysite/
* Make sure to delete the install folder on your live website!

5. MAKE A CHANGE LOCALLY AND THEN GET THE CHANGE ONTO THE LIVE SITE
-------------------------------------------------------------------------------------------------
It's good practise to NEVER make any changes to your live website. Rather, make changes to files on your local machine, use htt://localhost/mysite to test it and finally then ftp the changes up to your Webhost. We'll work through two examples, the first does not make any changes to the database and the second will have a database change meaning you'll have to also ftp a database dump up to your WebHost.

Firstly, I like to keep a track of which commit I've put into production
* Save the attached prod.log file into wamp/www/mysite/
* Run: git add prod.log; git commit -m "Log file to keep track of what versions were put in production"

Example 1 (no database change):
Modify stylesheet.css to remove website "brick" background
* Edit catalog/view/theme/theme439/stylesheet/stylesheet.css
* For me - I comment out Line 63 (i.e. make a minor change in your own stylesheet.css):
* Ensure this is the only file you change, run: git status
* Now commit your change, run: git add ., git commit -m "Removed brick background"

* Now we're ready to ftp this change to the Webhost, but let's first log the commit version:
* Run: (see commands in prod.log)

* Open up your ftp tool, like filezilla, do a diff between C:\wamp\www\mysite and your WebHost
* Ftp the difference (only)

Example 2 (with database change):
Delete a redundant currency type (ie data changes in db)
Right, the OpenCart files on your local machine are exactly the same as the files on your Webhost (with the exception of the config.php, /admin/config.php and new .htaccess files). We are going to remove a redundant currency via the admin panel on our local machine, then we're going to use git to verify that there are no file changes - - which obviously means all the changes must have been database based:

* To verify there are no uncommitted changes at the offset, run: git status
* Go http://localhost/mysite/admin > System > localisation > Currencies and delete one redundant currency
* Take note that on http://localhost/mysite your delete currency is no longer visible via the frontend
* To verify there were no file changes after the currency deletion, run: git status

So now we know those changes must exist in the database. To get this "currency deletion" change into production, all which needs to be changed is ftp'ing a dump of the localhost mysql database and importing it. so:
* http://localhost/phpmyadmin/ > Databases > Click db name > scroll down > tick "Check All" > Export > Custom
* For Custom:
<> unless otherwise stated I leave the defaults as is, but
<> ticked Include a timestamp of when databases were created, last updated, and last checked
<> ticked Enclose export in a transaction
<> ticked Add DROP TABLE / VIEW / PROCEDURE / FUNCTION / EVENT statement
<> ticked Add CREATE DATABASE / USE statement
<> ticked Truncate table before insert (!!!)
<> clicked Go
* Now ftp this file to your webhost, login to your webhost and import this sql over your existing database
* Your currency should now be changed also in production

------------------------ END OF EXAMPLES ------------------------

Be Careful with the live DB! Your live database is sacred. Always back it up BEFORE doing ANYTHING database related. You never want to loose the contents of your live database. This is how I make sure I maintain the integrity of my live database:
(1) Ensure my live and test environments have the same code;
(2) Export my live database and import into my test database on my PC;
(3) On my local test site, do and test out whatever change I need to do
(4) Check both changed code files and exported test database (ie the sql file) into git (as well as appended prod.log)
(5) Upload changed code files and sql file to live environment, import sql file into live database
(6) Do a final check on the live site

Gotcha! Initially after updating my live DB I went to my live site and still saw the currency I deleted. No problems I thought, I'll clear the browser cache. I did but I believe I still saw the deleted currency on my live homepage. I then logged into the admin panel > settings > localisation > currency to verify my deleted currency was not there - it wasn't. I then went back to my live homepage, reloaded, and what do you know - the currency was gone. Perhaps my observation was incorrect - but just watch for this strangeness!

Cheers,
Michelle (please let me know if you see any mistakes!)

Why would we need to use your email address and username? I think you may want to edit that bit.

SXGuy wrote:Why would we need to use your email address and username? I think you may want to edit that bit.

Thanks SXGuy, fixed

Sorry, but why so complicated??
Simply use this: TortoiseSVN

Install it on your machine locally having a full version control.
No command lines, simple and easy to use.
Callable by many editors like a normal SVN- or Github-repo.

OSWorX wrote:Sorry, but why so complicated??
Simply use this: TortoiseSVN

Install it on your machine locally having a full version control.
No command lines, simple and easy to use.
Callable by many editors like a normal SVN- or Github-repo.

It's just my preference for a terminal and distributed version control like git. I'm not saying this the "right" way. I'm just posting it in case there is even just one person this may be useful to :-)

Hi Michelle,
thank you for your long tutorial.I 've been tried to follow it step by step and surely it was help for me, to better understand something more about GIT and how we could use with Opencart development. On the other side, I'm still not completely convinced about the choice and the best way to use it.
I'd like to understand for example how I could use it ignoring all Opencart core files (not to be modified in general) and track changes/versions only on new files or vqmod scripts. Maybe a specific gitignore files could be used to ignore everything with only some exceptions.
Let's suppose to desire to design a new extension. Let's suppose to decide, as design rule, to name all involved files with a same prefix/naming rule, e.g.

myext_zzzzz.xml in vqmod/xml
myext_yyyy.php or .tpl in several other folders.

May I use a .gitignore like this?

# exclude all
/*

# not exclude specific extension vqmod/new files
!*/*myext*.*
!*myext*.*

It doesn't seem to work... any clue?

Thanks,

M

playing more with git. Maybe this .gitignore could be ok to my purposes:

#ignore all
*
# not ignore folders
!*/

# not ignore my ext files
!*myext*.*

Then, the general work flow could be:
At the beginning (if some myext files already exist, and need to develop/add new features):
git init
git add .
git commit -m "First commit - last stable release"

then, for each relevant change:
git add .
git commit -m "Description of change"

When a new release is ready,
git archive -o myext.zip <last commit>

What do you think about? What about branching?

M

Hi Madimar, I'm really happy this was useful for you. Apologies for not replying sooner.

I can not imagine not working with version control, even if it is just for myself and not in a team of people. Admittedly I previously used tortoise subversion and then moved over to git simply because it was new and "trendy" and called a "distributedversion control" system which I believe makes it much simpler for a group of people working on the same code. Rather than give my opinion:
* google "Git verse Subversion"
* Why is git better than subversion?
* Git Subversion Comparison
* Getting Started with Git Basics

As for branching - I have no idea. I work out the answers (much as you have) on a need to know basis otherwise I get overwhelmed. Because I haven't worked with a group of people for years and years on code - - I have no idea about branching. I have never needed it. I try to keep things as easy and simple as possible. I always believed there was a right and wrong way to do things, now I just use what works for me and then refine and make it better as I find inefficiencies along the way. And it appears you have something that works for you now too . I'm really not an expert in git at all, I just work it out and then try it out. Once I've got it I write it down to save myself from having to work it out again in the next year or two when I look at source code again

All the best and again I am very happy my post was useful to you. Michelle

Thanks Michelle anyway for your dedicated time and to share your experience.
Branching is import imho, to have a control during the time of different changes/implementations.

Example:
1) I have a stable version of a mod
2) I begin to start developing a new feature
3) Someone contact me for a bug to be fixed in the meanwhile

etc.

With branching (and maybe I figured out it is not complex with git), you can have:
1) the stable version on your master branch
2) Create a new branch for the new feature
3) Create a new branch for the bug fix
Then merge the branch with master when you finish one of the developments.

IT shouldn't be so complex, using:

- create a new feature branch and begin to work on it
git checkout -b feature_x

- go back to master
git checkout master

- merge a branch with another:
git merge <branch> (merging on the active branch, e.g. master)

Regards,

M