This is great at first but after a while log files become enormous and eat up lots of disk space. The problem multiplies when you start getting into a clustered environment (more than 1 server). You now have to read & maintain logs on 2+ machines which will most likely lead to neglecting your error fixing duties. Fortunately MongoDB recently hit the scene.

MongoDB

MongoDB is a scalable, high-performance, open source, document-oriented database. It’s ability to accept huge amounts of data very quickly make it perfect for application logging. It has built in support for auto-sharding & replication so when your application takes off mongodb will scale with you. It uses a JSON based storage system meaning you can give the database an object (from any programming language with support) and when you pull it back out of the database it will be the exact same as when you put it in. No conversion, no hassle, no bullshit.

Capped Collections

Mongo uses things called Collections which are essentially tables in mysql. The main difference is these collections are schemaless. I can have 10 fields in one document (row) and 500 in another and they will play nice inside the collection. This allows a huge amount of flexibility to the developer.

These collections also have the ability to be “Capped”. This means I can set a max document count of 5000 and only the most recent 5000 documents will be kept. The older documents are deleted as new ones come in. This is awesome for logging because you will never have to rotate your logs in fear that they will become too large and occupy all of your file system space.

Use this command to cap a collection (in a mongo shell):

db.createCollection("my_collection", {capped:true, size:5000})

More Information on Capped Collections

Use in CodeIgniter

By default CodeIgniter writes to log files in /system/logs/ . I have created a class that overwrites that default functionality and sends all logs to a mongo db collection. Adding this library to all of your servers will allow all of those servers to write to the same mongo collection. I have also added some more data to the logs that is helpful in a clustered environment such as the server name, server ip, request uri, and a few more. Just place the MY_Log.php in your application/libraries directory and codeigniter should handle the rest.

You can view and fork the class on Github:
http://github.com/trs21219/codeigniter-mongo-logs/

Feel free to modify data inserted to fit your specific needs. If you find any bugs or have suggestions you can comment below or raise an issue here.

Probably the biggest challenge for a newly popular site these days is scalability. The most common point of failure in applications seems to always be the database. These days our applications are totally reliant on some sort of relational database to store the information for our website’s content. Each and every page load brings many queries that read or insert information into this database all while rendering the web page in a timely manner. This isn’t so much a problem for your Mom & Pop business, but when your website starts to get some real traffic pointed towards it,  your underlying architecture that you thought was impervious to anything will crumble before your eyes. The solution? Stop making so many queries to your database.

Caching

Caching is nothing new. Servers have been caching content for many years quite successfully. There are many different ways you can cache data in your application but right now we will just be talking about using Memcached. Memcached is a high-performance, distributed memory object caching system. So what does this mean? Basically it can store any kind of value that you would normally store in your database and retrieve it much much faster than your database can.

Memcached Flow

When you implement memcached you use it along side of your database. Neither takes the place of each other, memcached just takes some of the load off of your database server after your database server has previously done the grunt work. It works on a key pair system. Meaning it has no tables, rows, columns etc. You give memcached a name and pass it data and it stores that data just as you passed to it. You can pass it an object, array, binary, etc and it will output the same way when you go to retrieve it.

So how does this all work with the database? First, you add code that checks to see if the memcached object (blogpost-45 for our example) exists before it checks the database for the data. If it does you return that data and never touch the DB.  If the memcached object does not exist  you retrieve data from your database, take the array that is passed back and store it in memcached with the key of “blogpost-45″. So the next time you try to get the data for blogpost-45 it will return from memcached and not your database. This is where the load savings occurs.

So what happens when I want to change the data? When you change the data in your database you should delete the key in memcached. You shouldn’t re-add the data after you delete it, let the select query do that to avoid complications.

CodeIgniter + Memcached

I have created a CodeIgniter Library that can help you get a jump start on your integration with memcached, its available on my github page and will be updated when I get time.

http://github.com/trs21219/memcached-library

You see when you are working with a multi-environment setup that is synced via a version control system such as Git or Subversion, you need a way to keep your development environments locked down while allowing access to your public environment.

Below is some text that you will input into a .htaccess file placed in your webroot

[sourcecode language='php']
#allows a single uri through the .htaccess password protection
SetEnvIf Request_URI “/testing_uri$” test_uri

#allows everything if its on a certain host
SetEnvIf HOST “^testing.yoursite.com” testing_url
SetEnvIf HOST “^yoursite.com” live_url
Order Deny,Allow

AuthName “Restricted Area”
AuthType Basic
AuthUserFile /path/to/your/.htpasswd
AuthGroupFile /
Require valid-user

#Allow valid-user
Deny from all
Allow from env=test_uri
Allow from env=testing_url
Allow from env=live_url
Satisfy any
[/sourcecode]

so in the above code the “testing_uri” part could be if i only wanted to allow this url through my htaccess protections (useful for paypal pings) ex “http://mysite.com/paypal/ipn”

the host part is to allow anyone through if they are requesting the code from a specific domain such as “testing.yoursite.com”. if it is “development.yoursite.com” it will not allow the user through.

here is the code on css-tricks and here is the code as a github gist

Thanks for reading and as always comments & questions are heavily encouraged!