Django community, search, security, performance, handling sloppy code — DjangoCon 2014

Here are some additional notes and thoughts from a few of the talks I attended at DjangoCon.

Daniele Procida keynote

Daniele Procida, one of the Django core team members gave a talk about how he ended up in such a position, having coming from a non-technical background. His conclusion was ‘All you need is a little l***,’ the key being the luck of surrounding factors rather than ‘hard work.’ Although I agree with him to an extent, I think he was confusing ‘hard work’ with ‘working long inefficient hours.’ However, his point about luck and situations is true to an extend, as might be supported by Malcolm Gladwell’s Outliers.

However, I think Procida’s main point was that there are high barriers of entry in the development world which involve luck. This isn’t ideal for an open source project such as Django. Hence we should aim to keep the barriers of entry low, so that more ideas and minds can contribute.

From __icontains to search

This went behind the scenes about some search logic works as __icontains is not ideal for many situations.

• Search is a problem that’s been solved before. An example is friars who indexed the bible to easily be able to reference passages during discussion amongst each other.
• Use an inverted index as a data structure for search.
• Stem words for the inverted index. It sounded a lot like some of the machine learning/prediction we are doing.

Secuirity (sic)

An overview of ways to prevent security issue by Levi Gross (some additional good tips on his site).

• Always be conservative when it comes to security.
• You’re app is only as strong as the lowest common denominator, especially if other parts of the app rely on each other.
• Explicit is always better than implicit. Ensure you know what type of input is coming in.
• Mixing up data and code leads to injects.
• Use security controls used by a large number of people. You’ll find strength in numbers and holes will be quickly fixed once discovered.
• Use the built in Django tools, as it follows the suggestion above.
• Ensure object permissions. What objects should have access to what other objects?
• Use Keyczar. And stay away from all encryption on your own, unless you really really know what you’re doing.

Development with Ansible and VMs

Ansible as an alternative to Chef? It looked fairly easy to setup and uses python. Although, I’m not sure how much easier it would be with an environment to setup like Hearsay Social’s.

Performant Django

A number of tips to improve performance.

Frontend:

• Cache static assets.
• Bundle & minify.
• Use data URIs to reduce the number of requests (this is an interesting one to me).
• Use CDNs.
• Store more things as static assets. One page apps are an example of this, where most of the time the data you’re passing back and forth is just json objects.

Backend:

• Use cached sessions.
• Use a cached template loader.
• Jinja2 is a pretty fast templating system.
• Use tools like profiler middleware to see what’s taking a long time.
• Debug toolbar & django-devserver
• Other methods to consider using, although not necessarily always the best are select_related, values_list, prefetch_related, only, defer, bulk_create.
• raw as a last resort to write your own sql.
• Denormalization
• Non-relational databases
• Bad algorithms or slow functions. Sometimes the same thing is being re-evaluated more than it needs to be, so maybe create a constant for it?
• Caching. There are many points at which you can cache things.

That’s a lot of tips, but definitely some good ones to keep in mind. Although you might not want to implement a number of them unless you have a specific need to resolve performance issues.

Inheriting a Sloppy Codebase: A Practical Guide to Wrangling Chaotic Code

This was a pretty cool one by Casey Kinsey. Feel free to go and bug him to refactor spaghetti code for you! And although Hearsay Social’s codebase is not as bad as it could be, a lot of the ideas definitely struck a chord with me and some of my recent projects involving refactoring.

Bad codebases come from:

• Inexperience
• Lack of long term ownership
• Emphasis on development speed, which happens often when ‘prototypes’ are pushed to production. It doesn’t work that way with prototypes in any other field.

How to handle it:

• Have a good IDE where you can jump around easily, refactor easily, and some code introspection.
• Make use of debuggers, like ipdb.
• The biggest point which was brought up over and over again by him, which I’ve written about before is TESTING! Write lots of integration tests. That way you’ll know when you break anything.
• Write unit tests for critical core components, but not necessarily on things that will be removed in the future.
• Remove as many dependencies as possible, especially unsupported external ones.

The main takeaway from this talk was write lots of tests before you attempt to refactor.

Anyways, those are just some of talks and notes from DjangoCon Day 2.

Originally published at newtonry.tumblr.com.