Probably the biggest challenge for a newly popular site these days is scalability. The most common point of failure in applications seems to always be the database. These days our applications are totally reliant on some sort of relational database to store the information for our website’s content. Each and every page load brings many queries that read or insert information into this database all while rendering the web page in a timely manner. This isn’t so much a problem for your Mom & Pop business, but when your website starts to get some real traffic pointed towards it,  your underlying architecture that you thought was impervious to anything will crumble before your eyes. The solution? Stop making so many queries to your database.

Caching

Caching is nothing new. Servers have been caching content for many years quite successfully. There are many different ways you can cache data in your application but right now we will just be talking about using Memcached. Memcached is a high-performance, distributed memory object caching system. So what does this mean? Basically it can store any kind of value that you would normally store in your database and retrieve it much much faster than your database can.

Memcached Flow

When you implement memcached you use it along side of your database. Neither takes the place of each other, memcached just takes some of the load off of your database server after your database server has previously done the grunt work. It works on a key pair system. Meaning it has no tables, rows, columns etc. You give memcached a name and pass it data and it stores that data just as you passed to it. You can pass it an object, array, binary, etc and it will output the same way when you go to retrieve it.

So how does this all work with the database? First, you add code that checks to see if the memcached object (blogpost-45 for our example) exists before it checks the database for the data. If it does you return that data and never touch the DB.  If the memcached object does not exist  you retrieve data from your database, take the array that is passed back and store it in memcached with the key of “blogpost-45″. So the next time you try to get the data for blogpost-45 it will return from memcached and not your database. This is where the load savings occurs.

So what happens when I want to change the data? When you change the data in your database you should delete the key in memcached. You shouldn’t re-add the data after you delete it, let the select query do that to avoid complications.

CodeIgniter + Memcached

I have created a CodeIgniter Library that can help you get a jump start on your integration with memcached, its available on my github page and will be updated when I get time.

http://github.com/trs21219/memcached-library

Recently at work, we realized that our SVN setup just wasn’t working anymore for us. Merges were difficult and time consuming, we had weekly release meetings to go file by file through a list of changes to make sure what we were pushing to production was actually quality. This process was severely inefficient and everyone hated it.  So we did what any company that absolutely hates svn does, we moved to Git & Github. We chose Github for the stability, function, and social coding features. We use freelancers fairly often to contribute to small modules we work on and having a rich GUI is absolutely essential.

Git-SVN

Now we had a years worth of commits that we wanted to migrate over to the new git system. I tried using the import tool that Github provides for repositories but due to our unconventional setup it couldn’t be done. After some searching  I found a command line tool that actually resides inside of the core git install called git-svn. With a few simple commands and about 45 minutes in the case of our repository it had converted every commit to the sha-1 equivalent that git uses.  Git-svn was actually designed so that you have the ability to work with both systems at the same time. So you could use git while your co-worker uses svn and both commit to the same repository. Below is the actual command we used to convert our SVN repo to a git repository. You can find the original tutorial on doing so here.

git svn clone --no-metadata -A authors.txt -t tags -b branches -T trunk

This took around 35 – 45 minutes with our repository which had almost 6,000 commits in it so give yourself some time.

Git GUI & Bash

Unfortunately, not everyone in the office is running on a mac. Now this wouldn’t be so bad if windows had native support for ssh but it doesn’t so we had to find a third party solution to using git. Good thing for us is that the git gui & bash application is pretty feature rich and has a low learning curve. After you install git via msysGit, head over to the github learing page or the progit book site to get your self acclimated with how git works.

Screencast on using git gui via Vimeo
[vimeo 2111264 400 285]

Line Endings

Let me just say this. Line Endings are a nasty bitch.  If you use multiple operating systems with your repository (especially windows) line endings can be a very big problem if you don’t set them up right from the start. What happens is that when Windows modifies a file and commits it it uses a different style of line ending than mac or linux. When your mac or linux machine pulls that down from the central repository, it changes the line ending thus changing the sha1 which makes git think you edited the entire file.  This bit us in the ass a week after converting our repo, which just so happened to be over the holidays so troubleshooting the problems it presented turned out to be especially hard. I highly recommend setting your line endings to the spec that this tutorial outlines. Since we did this we have not had line ending problems.

Workflow

We experimented with several workflows to see which would work best with Git. We landed on the “branch per task” solution. Basically whenever someone is going to complete a task ( a bug fix, feature enhancement, etc) they create a branch with the ticket name and number. They do their work in that branch, committing and pushing often. When their code is ready to be merged back into the master branch its a fairly easy task. We just run git merge origin/branchname and their changes are ported into the current codebase. We have found this best because it keeps developers from stepping on each others toes and its easy to delay a branch from making it into the master codebase if needed.

Server Deployment

We use several staging & production environments for alpha/beta testing  as well as our live environment. Pushing the code changes to these servers is actually quite easy. We just SSH right into the server, cd to the directory and run ” git pull “. Its as simple as that.

Overall we all like the new system. It took some getting used to and like everyone else we had our own WTF did I just do moments but in the end it works.

You see when you are working with a multi-environment setup that is synced via a version control system such as Git or Subversion, you need a way to keep your development environments locked down while allowing access to your public environment.

Below is some text that you will input into a .htaccess file placed in your webroot

[sourcecode language='php']
#allows a single uri through the .htaccess password protection
SetEnvIf Request_URI “/testing_uri$” test_uri

#allows everything if its on a certain host
SetEnvIf HOST “^testing.yoursite.com” testing_url
SetEnvIf HOST “^yoursite.com” live_url
Order Deny,Allow

AuthName “Restricted Area”
AuthType Basic
AuthUserFile /path/to/your/.htpasswd
AuthGroupFile /
Require valid-user

#Allow valid-user
Deny from all
Allow from env=test_uri
Allow from env=testing_url
Allow from env=live_url
Satisfy any
[/sourcecode]

so in the above code the “testing_uri” part could be if i only wanted to allow this url through my htaccess protections (useful for paypal pings) ex “http://mysite.com/paypal/ipn”

the host part is to allow anyone through if they are requesting the code from a specific domain such as “testing.yoursite.com”. if it is “development.yoursite.com” it will not allow the user through.

here is the code on css-tricks and here is the code as a github gist

Thanks for reading and as always comments & questions are heavily encouraged!