Heynemann – Just another dev

Introduction

MI7 is my new pet project. I grew tired of the other mocking/stubbing/spying engines in Python that never did quite what I expected them to.

I’ve been TDDing for a while now in .Net, Ruby, Python and JavaScript. I’ve got my fair share of experience, so I figured I’d give my 2¢ in this issue.

You can check the project at https://github.com/heynemann/mi7/wiki. It’s got a nice tutorial and is currently in 0.1.1 alpha release.

I’ve got a lot of work to do to make it my main tool for test support, but I’m going to get there. Without further delay, let’s get to it.

Why another Spying Engine?

If you check the MI7 wiki you’ll see that I don’t have anything against any single python test support engine. I just haven’t found one that suits my needs and those needs ONLY. IMHO they all do too much. I want a simple, straightforward, fun to use spying engine.

Don’t get me wrong, but I do not believe in mocking in Python. Or stubbing for that matter. Both are akin to dependency injection, IMHO. It just isn’t pythonic.

Python has been around for a while. In this time, there has been a certain Modus Operandi of doing work. This MO has never included injecting your dependencies around. I figured that’s why I feel the weirdness on the part of the mocking/stubbing/spying tools.

With MI7 I’m trying to interfere as less as possible with your code. Production code should be optimized to be production code, and not changed to accommodate your poor testing tools. In the Ruby community they try HARD to make tests and code as clear as possible, not make code work according to tools. I’m trying to get some of that.

The last point of me doing MI7 is to have some fun, and I’m trying to bake in the library as much of that fun as possible, with the spy agency metaphor. Hope you enjoy as much as I am.

Test Sample

Ok, so I’ll write a test with MI7. It’s pretty simple:

from controllers import MyController
from models import User
@new_mission
@agent.spy(User)
def test_user_is_authenticated():
    agents.User.intercept('is_authenticated') \
               .returns(True)
    agents.User.intercept('username') \
               .as_attribute('Bernardo')
    ctrl = MyController()
    result = ctrl.index()
    assert result == "Welcome Bernardo"

So what’s happening here. I’m telling MI7 to keep an eye in the User model, wherever it may be used. Then I’m instructing the User agent (the agents get their code-name from the target they are spying), to intercept calls to is_authenticated and username and return my values.

Now the controller code:

from models import User
class Controller(object):
    def index(self):
        user = User()
        if user.is_authenticated():
            return "Welcome %s" % user.username
        return "Unauthorized"

As you can see, there's not a single line of code in that controller that says "I'm testable". It's just plain old python coding.

Current Status

Currently MI7 supports intercepting modules and classes and telling agents to intercept methods and attributes and to raise exceptions.

Impersonation (stubbing) may come next. Definitely assertions are coming, like what an agent has seen and such.

Conclusion

I’ll keep going with MI7 development as much as I can, because I believe the Python community needs better testing tools and I’m willing to put extra effort into this.

The Problem

Nowadays with all the JS frameworks and CSS Frameworks, our web applications end up with a very rich and interactive UI (also called User Experience). With the easy-of-use that this tools bring, many plug-ins and third-party code gets integrated in our apps. Usually this code comes bundled with many CSS and JS files (usually one for each plug-in). They easily pile-up to become a bottleneck of browser rendering performance.

As Yahoo points out in this article, one of the best ways to speed up your website is to drastically reduce the number of requests that the user’s browser has to do.

Another way to improve performance is to minify your CSS and JS. The minify action reduces the number of bytes that go through the wire, thus improving download speed for your users.

At the company I work for we faced a big issue with our pluggable app. After many plug-ins and customizations by clients, our pages were REALLY big with MANY (100+) requests. You can easily guess that this is not a situation you want your website to be in, right?

Django Compressor

Enter Django Compressor, the pluggable app for Django that provides a template tag to compress CSS and JS. It does more than that, though. Django Compressor provides an extensible architecture that allows you to implement your own template parser, your own template filters and your own static CSS and JS Storage. We’ll discuss more of each in the following topics.

I just want to thank beforehand all the effort that has been put in this project and all the involved people. The quality of the project has really surprised (a good surprise) me and I’m really glad to have found it (via Gabriel Falcão).

Basic Usage

As you can read in the readme file (or in github main page), it’s really simple to set Django Compressor up. The steps are:

1. Add ‘compressor’ to INSTALLED_APPS in the settings.py file for your Django app (make sure compressor can be imported using ‘import compressor’ in python).
2. You don’t *HAVE* to, but I like to specify whether I want compression on or off (the default is the opposite of DEBUG, meaning that in DEBUG mode you don’t compress and with DEBUG off you get compression). If you, like me, want to specify whether to compress or not, just add COMPRESS=True to your settings.py file (anywhere).
3. Go to a template where you have many CSS and include the following:

   1. {% load compress %} before using any compression.
      This loads the template tag.

   2. In your templates, you compress your stylesheets:
      {% compress css %}
      <link type="text/css" rel="stylesheet"
            media="screen"  href="/media/my.css" />
      <!-- many more css files -->
      {% endcompress %}

That’s all there’s to it. After reloading your view you should see that the my.css link got replaced with <<some_hash>>.css. The hash here is generated based on the contents that got compressed. This means that all the CSS requests that were being done previously will now be done in one go. The same goes for JS, the only difference being that you supply ‘js’ as argument to compress, like this:

{% compress js %}
<script type="text/javascript" src="/media/my.js" />
<!-- many more js files -->
{% endcompress %}

It gets better, though!

Compressor Filters

Django Compressor uses a really simple and easy to understand filter architecture. Basically whatever filters you specify for JS (in the COMPRESS_JS_FILTERS setting) and for CSS (in the COMPRESS_CSS_FILTERS setting) will get called sequentially in order to modify the output.

What this means is that if you want to validate, minify and then concatenate your CSS files, all you have to do is specify the proper filters (or implement your own).

In our current scenario we are using the following settings:

COMPRESS_CSS_FILTERS = [
     'compressor.filters.cssmin.CSSMinFilter'
]
COMPRESS_JS_FILTERS = [
     'compressor.filters.jsmin.JSMinFilter'
]

As you can see I’m only using the filters that are already provided with Django Compressor. What this means is that I get for free CSS and JS minification.

If you want to implement your own filters I suggest you take a look at the code for the current filters in Django Compressor (in the filters folder in the source code).

Django Compressor FileStorage

Django Compressor uses Django’s concept of a File Storage. A file storage is an entity responsible for every related file operation that Compressor needs to execute (verifying if a file exists, building file paths, opening and saving files, etc.).

Here at our team we have media distributed in many apps, all of them inside a /media folder. We distribute media like that because our clients plug-in only the apps that they need, so it makes sense that the medias stay where they actually belong. That works pretty well with the way django loads media but deployment is a little harder. We had to work our own deployment scheme (subject for a later post).

By default django compressor only uses the projects media folder (as specified in the MEDIA_PATH setting). We had to change it so we could use it with our apps. It was as easy as implementing our own File Storage class. You can check it at my fork in http://github.com/heynemann/django_compressor. As of the time of this post our code has not been integrated to the Django Compressor project. We are hoping that it does soon.

To use our custom storage all we had to do was specify it in the settings.py file with:

COMPRESS_STORAGE = 'compressor.storage.AppSavvyCompressorFileStorage'

And that’s it! Now the pages that refer to medias in our apps also get compressed.

Conclusion

Django Compressor is a really cool project and very worth contributing to. It already features many filters, parsers and now two storages. If you want to speed-up the loading time of your pages and you are using Django, you definitely should consider it.