We’re pleased to announce a new Python instrumentation version — Oboeware 1.1!

We’ve added a few new libraries recently, but we’re really excited about the new customization API we’ve introduced in this version. More than just a Python bump, this is the first package we’re releasing with an implementation of our new Oboe API. The Oboe API is a common set of idioms and metaphors for extending Tracelytics instrumentation or quickly writing your own from the ground up. We’re excited to get it out there, and we’re even more excited to see what you build with it!

Conceptually, the Oboe API is a multi-tiered system that allows instrumentation of everything from simple function calls to crazy distributed asynchronous event-driven applications. There are three parts: the low-level API, the high-level API, and language-specific functions.

Language-specific functions

The language-specific functions are just that: language-specific idioms that give you the most tracing bang for your buck. In Python, we’ve put together 4:

• The @trace decorator allows you to start a trace wherever you want. While we catch most web requests, sometimes it can be valuable to trace your backend processes as well. This can give you visibility into celery jobs, command-line scripts, or cron jobs.
• In an existing trace, the @log_method decorator creates a new layer. We use this extensively in oboeware to actually instrument the standard python libraries like urllib2, pylibmc, or pymongo. If you’ve got an internal API client, or even just a wrapper around an existing library, use this to make sure you’re seeing exactly how performant that library is!
• The @profile_block and @profile_function allows you to mark particular blocks of code in your app, and we’ll keep track of them in Tracelytics. If you’ve got a particularly thorny algorithm, or just a lot of log parsing to do, throw it inside a @profile_block…

  log = open('log.txt', 'r')
  
  with oboe.profile_block('log_parsing'):
      # Unzip, read, segment, analyze, understand, ...
  
  log.close()
  

  … and make sure you’re never losing sight of how long that job is taking.

We understand that modifying your code with performance annotations can be trying. These functions aim to be one-line modifications that open up whole new areas of visibility in your app, with no change in functionality.

High-Level API

Do you want to trace your custom-built, high-performance, NumPy simulation library? Are you looking for a simple way to instrument your internal-but-open-sourced RPC protocol? Or do you maybe want to trace all 7 custom tiers in your in your 7-tier web app?

The high-level API provides a logging-like set of functions that report to the Tracelyzer agent. With these functions, you can control exactly what, where, and how you report your performance information. Additionally, this lets you control the key/value pairs reported, which can be used to mark certain layers for special treatment in Tracelytics. For instance, adding the KVOp key identifies the layer as cache. Adding these keys to a fictional hybrid Cassandra / memcache cache would allow visualization in the same place.

oboe.log_entry('memssandra', { 'KVOp': 'get'})
try:
    return cache.get_value(key)
except Exception:
    # log exception and re-raise
    log_exception()
    raise
finally:
    oboe.log_exit('memssandra')

Layers introduced by the high-level API aren’t an afterthought — they’re treated exactly the same as any other layer within the Tracelytics interface. Each layer gets its own visualization and a set of filters, and you can set Alerts on it just as you would full requests or application times.

Low-Level API

For the truly hardcore, we also have a low-level API. In addition to all of the power and flexibility of the High-Level API, this level adds the concept of a Context — an object that encapsulates the request context needed to trace a request through your full stack, under whatever evented, microthread, or serializable paradigm you’re working.

Build it out!

We hope this release is the start of a long and varied journey to tracing everything under the sun. Are you building something with it? Let us know! Are you thinking about it? Sign up for a Tracelytics account, and get started today!

Over the Labor Day Weekend, I spent my free time hiking New Hampshire’s Presidential Traverse. The Presi Traverse is over 20 miles long, covers 7 mountains over 4000 ft, has a total elevation gain of 7800 ft and a total elevation loss of 7400 ft. The entire hike took my group 14.5 hours.  Not to outdo ourselves, due to weather conditions and the full moon, we started the hike at 5 PM, hiked through the night and finished at 7:30 AM!

Along the way, I had some time to think. My thoughts wandered to how this hike compared to full-stack application monitoring (I know, it’s the weekend and I’m thinking about APM solutions). Here’s how:

Each mountain I climbed is similar to the different layers that make up a web application. There is a path between each mountain that I followed from start to finish which is similar to the path a web request takes within the web application. And, if any problems arose, I was carrying a headlamp that I could turn on at any moment (thanks to the full moon, this was not needed for most of the hike). This is similar to an installed APM solution. Both are designed to consume less that 1% of overhead, are there when you need them, and are out of the way when you don’t.

A Tracelytics View of the Presidential Range

If I examine the entire hike from start to finish, I can produce averages of elevation gain, hiking pace, and total time to complete the hike. The details are explained in the second paragraph above. These averages are similar to partial-stack application tracing. I see the data, but have no way of pointing out problem areas, getting detailed data or reporting on paths between layers.

When I break my trip down into the below stats, I return data that is more consistent with full-stack application monitoring. For each mountain (layer) and trail (path), I have data on time to complete each leg (web request), the name of the mountain (language or operating system), and distance/gain/loss/total time (length of request).

Now, if you finished reading this post, and you’re not as exhausted as I was at the end of my hike, sign-up for a FREE 14-day trial of Tracelytics. Our SaaS solution installs in minutes and performance data is revealed in seconds!

So here are my “full-stack” details (all photos from Eric Bauer):

2.4 miles in - 15.2 miles to go

• Appalachia to Mt. Madison
  • Start: 5 PM
  • Finish: 7:15 PM
  • Distance: 4.25 miles
  • Elevation start: 1303 ft
  • Elevation end: 5367 ft
  • Total gain: 4000 ft
• Mt. Madison to Mt. Adams
  • Start: 7:30 PM
  • Finish: 8:30 PM
  • Distance: 1.3 miles
  • Elevation start: 5367 ft
  • Elevation end: 5799 ft
  • Total gain: 700 ft
  • Total loss: 260 ft

Sun goes down - headlamps turn on

• Mt. Adams to Mt. Jefferson
  • Start: 9 PM
  • Finish: 10:30 PM
  • Distance: 2.4 miles
  • Elevation start: 5799
  • Elevation end: 5716
  • Total gain: 800
  • Total loss: 860
• Mt. Jefferson to Mt. Washington
  • Start: 11 PM
  • Finish: 12:30 AM
  • Distance: 3.2 miles
  • Elevation start: 5716 ft
  • Elevation end: 6288 ft
  • Total gain: 1300 ft
  • Total loss: 750 ft

6288 ft above sea level - 12:30 AM

• Mt. Washington to Mt. Monroe
  • Start: 1:30 AM
  • Finish: 2:45 AM
  • Distance: 1.8 miles
  • Elevation start: 6288 ft
  • Elevation end: 5372 ft
  • Total gain: 360 ft
  • Total loss: 1270 ft
• Mt. Monroe to Mt. Eisenhower
  • Start: 3:15 AM
  • Finish: 4:15 AM
  • Distance: 2.8 miles
  • Elevation start: 5372 ft
  • Elevation loss: 4780 ft
  • Total gain: 300 ft
  • Total loss: 900 ft

Finally, the end 7:30 AM

• Mt. Eisenhower to Mt. Pierce
  • Start: 4:30 AM
  • Finish: 5:15 AM
  • Distance: 1.7 miles
  • Elevation start: 4780 ft
  • Elevation end: 4310 ft
  • Total gain: 300 ft
  • Total loss: 780 ft
• Mt. Pierce to Highland Visitor Center
  • Start: 5:45 AM
  • Finish: 7:30 AM
  • Distance: 3.1 miles
  • Elevation start: 4310 ft
  • Elevation end: 1909 ft
  • Total loss: 2400 ft

And now for the exciting news… On September 6, 2012 at 12PM EDT, AppNeta is hosting a kick-off webinar to talk about key challenges with Web application performance and hear from one of our very own customers about the success they have had with Tracelytics! The webinar will cover the following topics:

• Common Drupal performance pitfalls and solutions
• Key performance indicators to keep an eye (and alert) on
• How to use Tracelytics to get quick answers to performance problems

If you’re a Drupal user, this webinar is a great opportunity for you to tune in to learn how you can use Tracelytics to manage your Drupal app’s performance. If you’re not a Drupal user, we still want to hear from you and would be happy to do a 1:1 live demo – AND FREE TRIAL – of the service!

I look forward to having you join us.  Sign up here for the webinar on September 6, 2012 at 12PM EDT.

Interested in getting started with Tracelytics today, sign up here for the free 14-day trial.

Service Oriented Architecture (SOA) seems like a vague, fuzzy term. At its core, SOA means separating software into isolated, reusable components (“services”) that are accessed through well-defined interfaces. It emphasizes loose coupling, so that instead of directly calling into another component (like when using a shared library, static linking, etc.), components communicate using a network protocol.  In practical terms, this means that changes can be made to a service without affecting its clients (“consumers”), as long as the protocol remains compatible.

It is an “architectural style”, not an implementation. There are many ways to implement SOA, including REST and SOAP web services, language-specific implementations such as Java RMI, and older approaches using RPC frameworks such as CORBA (which dates back to the 90′s!).  Some folks also decide to build their own service / RPC layer on top of messaging frameworks such as Protocol Buffers or MsgPack. Others may even go as far as implementing their own custom network protocols at the wire level. There is also Thrift, which we use here at Tracelytics.

SOA with Modern Web Applications

Many web applications start off as a monolithic code base, then move to a more distributed system using SOA. One example I’ve seen at another company: an application that generates reports from search engine marketing data. The original web application was written in PHP, with all processing, database queries, etc. occurring synchronously in the context of the HTTP request. Large reports could take several minutes (and, sometimes, hours!) to complete, and users would become impatient waiting for their browser to finally display the result.  To improve the user experience, it was decided that reports would be run asynchronously, in the background, which allowed users to check on their status, receive emails when they were finished processing, archive the report for later, etc.

This was done by moving the report processing logic into a separate component: a SOAP web service implemented using JBoss, a Java application server. The front end web application remained in PHP. The PHP web app would send a HTTP SOAP request to the report service, requesting that a report be initiated. If there was too much load on the system (for example, more than N reports running in parallel), the report request would be queued for later processing.  The user could continue working with the application and would receive notice when their report was complete.  The system also performed better, since the new service code was more efficient than the previous PHP implementation, and the number of actively processing reports could be controlled, based on system capacity.

Also, it was a more effective use of development resources since two completely separate teams could develop the code: a front-end / PHP team, and a back-end / Java services team. These teams only needed to agree on the interface (in this case, the WSDL), which defined the method names, parameters, return values and any associated data structures.

How We Use SOA Inside Tracelytics

Tracelytics is also built on an SOA architecture. One example is our data-sharding service (internally known as “shardy”).  When the web application needs to retrieve certain types of data, for example the heat map / latency data for a particular customer app for a particular time range (last day, last 7 days, etc), it makes a request to the data-sharding service.  The service locates the data, which, depending on the time range, could be spread across dozens of tables and database servers (“shards”). It then queries it, running several SQL queries in parallel,  performing the necessary aggregation (such as filtering and sorting of the merged data set), returning a single result set to the application.

This service was developed using  Thrift, a framework for building services.  Though originally started as a Facebook project, Thrift has since found a home at Apache. It is cross-language, meaning a service can be written in one language and called into by another. It supports C++, PHP, Python, Perl, Java, and many other languages. Currently, our service and client (web app) are both written in Python, but this compatibility means that there is no reason this couldn’t change in the future.

Developing a Service using Apache Thrift

Before developing any type of service (whether with Thrift or not!), one should design the service interface. This involves specifying the service calls, which includes the service name, method names, parameters, data types, and return values in a generic, language independent way. With Thrift, this is done by writing an IDL (interface definition language) file. There are many tutorials on how to do this, so we won’t repeat the same information here.

The IDL file is then used as input to a code generator (“thrift -gen …”), which generates both client and server stubs for you.  Thrift takes care of a lot of the tedious work: you don’t have to worry about writing error prone network communications code.  Of course, the server stub will then need to be modified to actually implement your service (the “business logic” that actually does the work of your application ).

Tracing Performance Of Thrift Services

If you’re using a Thrift service, you must monitor the performance of that service to see how it’s affecting the rest of your application. Tracelytics allows you to do this by customizing your client code.   Internally, we’ve actually modified the Thrift code generator to make this automatic. Please let us know if you’re interested in this functionality!

(Photo Credit: southbeachcars)

A simple celery task

For a quick review, Celery lets you turn any python method into an asynchronous task.  Here’s a simple one:

from celery.task import task
@task
def add(x, y):
    return x + y

from celery.task import task
import oboe
from oboeware import loader

# load monkey-patched instrumentation for supported modules
loader.load_inst_modules()

# trace every unit of work done by decorated method
# (use fractional sample rate to trace a fraction of calls to decorated method)
oboe.config['tracing_mode'] = 'always'
oboe.config['sample_rate'] = 1.0

@task
@oboe.trace('celery', kvs={'Controller': 'task', 'Action': 'add'})
def add(x, y):
return x + y

The key part is the @oboe.trace decorator.  Note that we’re assigning to the keys Controller and Action.  This will be used by Tracelytics to segment the data.  You can also optionally use the keys HTTP-Host and URL to indicate domain and URL to Tracelytics.  Here’s what the data we’re gathering looks like in the dashboard (I added a few sqlalchemy queries to spice up the data):

The code at the top is configuration tunables: set the sample rate to a fractional value to trace only a fraction of your Celery tasks.

Event-based workers

You might be using event-based workers to save a few bytes of ram.  Tracelytics also supports eventlet-based workers.  However, you’ll need to install our Tracelytics-enabled greenlet module using pip or easy install:

$ pip install --extra-index-url=http://pypi.tracelytics.com greenlet-0.3.4-tly1.0

Slightly more interesting

Of course, you’re probably interested in more complex celery tasks than that.  The good news is, all of our normal instrumentation works with your celery workers.  Here’s a screenshot of one of our internal Celery workers in action:

In fact, if your app isn’t fast, it is absolutely costing you business – See the Amazon example where a delay of just a few miliseconds cost them 1% of their revenues. The truth of the matter is that with website speed, it’s not that you get brownie points for being faster – you get penalized for not being lightning fast. Here are 10 great reasons why you need to be obsessed with making your website faster:

1. Your consumers expect sites to load in under 2 seconds. When asked by Forrester Consulting how quickly they expected a web page to load, 47% of respondents answered 2 seconds or less. The concept of “loading” doesn’t exist anymore. People now expect browsing the internet to be like reading a book; they expect the next page to already be there. And they get annoyed if it’s not, because waiting for your page to load makes their brain hurt. No, really.

2. If your site takes more than 2 seconds to load, they’ll leave. Nobody is going to sit at their computer twiddling their thumbs and ruminating on how far human civilization has come since the first letter was carved into a slab of stone as the sun set on the Euphrates River. What they will do is forget about you; 14% will go to a competitor’s site, and 39% will start doing something else on their computer that distracts them, like looking at videos of teenagers severely injuring themselves doing skateboard tricks so they can feel better about spending their day behind a computer screen in an over-air conditioned room.

3. The difference between a user preferring a competitor’s site or preferring yours is 250 milliseconds. If a competitor’s site is just that much faster than yours, people will subconsciously notice it and your company will lose clout and users. You know what else happens in 250ms? Nothing.

4. Seriously, people won’t like you anymore. Remember that time you had a little too much at the Christmas party and made an ass of yourself? That’s nothing compared to this. 46% of dissatisfied online shoppers are likely to develop a negative perception of the company, and 44% of them would tell their friends and family about it. Which is unfortunate, because people tend to perceive page load times as 15% slower than they actually are, and 35% slower when they talk to other people.

5. Your search ranking will also suffer. Google takes website speed into consideration in their ranking algorithm.

6. Users continue to expect higher quality graphics, which means larger webpage sizes, even as they continually want quicker load times. Just an observation, one which has broad implications for your business; your site is going to have to keep getting stronger, because the amount it’s going to have to lift is only going up.

7. If you do have a website that’s up to par, users will spend more time on your site, lowering your bounce rate, and increasing your conversion rate and revenue.

8. If you’re not yet convinced that we all become spoiled children as soon as you sit us in front of a web browser, consider these anecdotes from big corporations. Amazon found that a 100ms increase in load time meant a 1% drop in sales—that’s a lot of money for so seemingly imperceptible an amount of time. That means 1 extra second of load time would cost them $1.6 billion.

9. In 2007, Shopzilla decided their site was too slow and that they would re-engineer it. They managed to decrease load time by several seconds. As a result, they get 25% more page views, revenue increased by 10%, and cut the number of servers they needed from 402 to 200.

10. The sin of sloth means a lot for search engines as well:

• Yahoo found a 400ms delay in load time decreased traffic by 9%.
• For Bing, a 2 second slowdown in results led to 1.8% less searches and decreased revenue per user by 4.3%.
• Google discovered that slowing down search results by just 100-400 milliseconds can induce users to perform .2%-.6% less searches. Even when the delay was removed, it took weeks before users returned to their previous search volume.  Yes, your users are that fickle.

That sounds like a lot of pain and misery if your site is slow.  If only there was some way to identify the sources of latency in your site or app by tracing requests through your entire architecture from the front end to the server. Then you could eliminate them, your customers would love you, your competitors would crumble at your feet begging for mercy, you would be less stressed, lose 10lbs, and finally be able to take a vacation.
If you’re looking for a secret weapon to help understand your web application’s latency and speed it up, try out Tracelytics free for 14 days. Don’t wind up like Don Draper.

- More than 260 repository watchers

- 24 active forks of pageguide.js ongoing

- Many user submitted patches & new features (and more coming in the works!)

- Now included in nuget and other packaging libraries to make deployment even faster.

- Great coverage in SmashingMag, Dzone, and countless other web design or web utility blogs, that drove even more great attention and awareness of our library.

Thanks to all of you since checking out pageguide.js and spreading the word – We really appreciate it, and we look forward to seeing it on more websites around the web soon. If you’ve implemented pageguide.js on your own site somewhere, please send us a link – We’d love to see how you are using it. Drop us a line at bwhalley@tracelytics.com.

AppNeta has built great technology and a fantastic product around network performance management and end-user monitoring. We’ve been focusing on providing insight into what’s going on in complex, distributed applications. And now, we’ll be the first company to provide truly complete, end-to-end visibility all in one place.

In March, we closed our Series A. With a great set of investment partners, our loyal customers, and everything that we’ve learned since we started back in June of 2010, we set off on the next stage of the Tracelytics journey. Our best efforts were not enough to predict where we would be just four months later.  It’s one of those occasions where aiming for the stars lands you in a distant galaxy.

We’re very excited about the future.  Today we’ve taken a big step towards our goal of tracing everything.

Loading Strategies

One of the most common pitfalls of using an ORM is accidentally creating a Query in a Loop. In SQLAlchemy, writing something like this:

# Assuming a User class, with a group relationship to a different table...

>>> users = Session.query(User).all()
>>> for r in users:
...     print user.group

This runs an initial query to get the list of users, then an individual query for each user to get the group! In many ORMs, you’d have to change the structure here: batch load the users, batch load the groups, then iterate over the loaded results. SQLAlchemy provides a better way, called Loading Strategies. Instead of changing the logic, you just specify which relationships to load, and SQLAlchemy will get everything you need in one query. In our example, we’d like to load all groups at the same time as the users.

>>> users = Session.query(User).options(joinedload('group')).all()
>>> for r in users:
...     print user.group

Fewer roundtrips, much faster, and hardly any code change.

Reconstructing Objects

One common pattern in OO designs is to lazily compute properties, cache them, and return the cached version after that. Not only does this have performance benefits, but it also keeps the computation logic for given properties separate and coherent. A simple method might look like this:

class Foo(object):

    def __init__(self):
        self._bar = None

    def bar(self):
        if self._bar is None:
            self._bar = ... # Compute bar
        return self._bar

This pattern unfortunately doesn’t work with SQLAlchemy. Specifically, objects loaded from the DB don’t re-run the __init__ constructor, so they won’t have the _bar property available, and the line self._bar is None will fail with AttributeError: 'Foo' object has no attribute '_bar' . To fix this, SQLAlchemy offers a hook that runs after loading: sqlalchemy.orm.reconstructor. In our case:

from sqlalchemy.orm import reconstructor

class Foo(object):

    def __init__(self):
        # Initialize persistent properties here
        self.init()

    @reconstructor
    def init(self):
        # Initialize transient properties here
        self._bar = None

    def bar(self):
        if self._bar is None:
            self._bar = ... # Compute bar
        return self._bar

In general, we do all transient initialization in a reconstructor method and only touch persistent properties in the constructor. Without that, new objects may behave differently than loaded objects!

Polymorphic Identities

ORMs typically model a single row in the database as an object, with each column represented as a field, and SQLAlchemy is no different. For most data, this makes sense. A “users” table with columns “first_name”, “joined_on”, and “active” maps cleanly to an object with the same properties. However, not all columns represent simple data. Something like “role” identifies a behavior on an object. Naively, the model methods that use that column might look like this:

Base = declarative_base()
class User(Base):
    __tablename__ = 'user'

    role = Column('role', String(20), nullable=False)
    # other column definitions here...

    def get_menu_dropdown_items(self):
        if self.role == 'owner':
            # return all choices
        elif self.role == 'admin':
            # return all choices, except delete organization
        elif self.role == 'base':
            # return just personal settings
        else:
            # return blank list

Instead of returning the column data itself, the object behaves differently based on the value of the column. A common OO pattern when faced with this kind of switch is to choose a different class for each (e.g.) role. By pushing this conditional “up a level”, to class instantiation time, we only have to make that choice once, instead of repeating this if/elif/else block across all the model methods that might need it. (If you want to read more about this pattern, check out the wiki article Replace Conditional with Polymorphism).

By default, SQLAlchemy picks the class type when it loads the rows. To change this, we can give the base class a polymorphic type — a row name that SQLAlchemy will use to figure out which class to instantiate, passed to the mapper. Each subclass can then declare a polymorphic identity — a row value to associate with the class. If you’re using the declarative base, you can set the __mapper_args__ to the appropriate column and value. For instance, the above example could become:

Base = declarative_base()
class User(Base):
    __tablename__ = 'user'
    __mapper_args__ = { 'polymorphic_on': 'role' }

    role = Column('role', String(20), nullable=False)

    def get_menu_dropdown_items(self):
        # default implementation: return blank list

class AdminUser(User):
    __mapper_args__ = { 'polymorphic_identity': 'admin' }

    def get_menu_dropdown_items(self):
        # return all choices, except delete organization

class OwnerUser(User):
    __mapper_args__ = { 'polymorphic_identity': 'owner' }

    def get_menu_dropdown_items(self):
        # return all choices

class BaseUser(User):
    __mapper_args__ = { 'polymorphic_identity': 'base' }

    def get_menu_dropdown_items(self):
        # return just personal settings

Now, querying the User class will actually return a number of different implementations, matched to the data!

This pattern is useful when you have a number of different types that you expect to have to change with some frequency. Obviously splitting up the get_menu_dropdown_items logic across multiple classes makes it somewhat harder to see all types of behavior. In return, implementing a new class (e.g., guest), is much easier, since all the methods are in one place, and can even be statically checked (for existence) by tools such as PyLint.

Lazy Loading Dependencies

In an ideal world, all models have no dependencies. It would even be fine to have models with one-way dependencies, with no cycles. Unfortunately, this isn’t always the case. Consider two classes, with mutual references:

from sqlalchemy import *
from sqlalchemy import orm
from sqlalchemy.ext.declarative import declarative_base

Base = declarative_base()
class Account(Base):
    __table_name__ = 'account'

    id       = Column('id', Integer, primary_key=True)
    prefs_id = Column('organization_id', Integer, ForeignKey('preferences.id'))

    prefs = orm.relation(AccountPrefs, uselist=False) # One-to-one relationship

class AccountPrefs(Base):
    __table_name__ = 'preferences'

    id         = Column('id', Integer, primary_key=True)
    account_id = Column('organization_id', Integer, ForeignKey('account.id'))

    account = orm.relation(Account, uselist=False) # One-to-one relationship

However, when we try to create this, it fails with NameError: name 'AccountPrefs' is not defined. This is because we’re referencing the AccountPrefs class before its definition. We can fix this by changing the relationship lines to the following:

    prefs = orm.relation("AccountPrefs", uselist=False) # One-to-one relationship
    # ...
    account = orm.relation("Account", uselist=False) # One-to-one relationship

Instead of the class instead itself, we can pass a string to relation and defer creating the relationship until the loader is first used — typically after the import has finished.

If these classes are sufficiently large, or have other support code, it may make sense to break them up into two modules (e.g., account.py and account_prefs.py). This introduces a new problem: if one of these modules is loaded without the other, SQLAlchemy will again throw a NameError. Deferring this loading forever won’t help — the class definitions simply aren’t available!

Fundamentally, we have to force both modules to load if either module is loaded. We could just put the appropriate import statement in both files, but importing modules without a clear use of any of the variables seems kludgy (remember that we’re hiding the relationship declaration in strings already — tools like pylint will issue warnings on these unused imports). Ideally, we’d have an explicit declaration, with a clear purpose. Fortunately, SQLAlchemy 0.7.7 introduces a new hook: __declare_last__. This allows us to execute a function after the relationships are loaded, but before they are used. This gives us a chance to nest the circular import:

# account.py
Base = declarative_base()
class Account(Base):
    __table_name__ = 'account'

    id       = Column('id', Integer, primary_key=True)
    prefs_id = Column('organization_id', Integer, ForeignKey('preferences.id'))

    @classmethod
    def __declare_last__(cls):
        from account_prefs import AccountPrefs
        cls.prefs = orm.relation(AccountPrefs, uselist=False)

# account_prefs.py

from account import Base
class AccountPrefs(Base):
    __table_name__ = 'preferences'

    id         = Column('id', Integer, primary_key=True)
    account_id = Column('organization_id', Integer, ForeignKey('account.id'))

    account = orm.relation("Account", uselist=False) # One-to-one relationship

We like this because it lets us manage different chunks of logic, even when that code depends on data that is naturally tightly coupled.

Even more…

There’s obviously a lot more to SQLAlchemy. What are some of your favorite features? Let us know in the comments!

Installation

Installing the Drupal module is as straightforward as downloading it and extracting the .tar.gz or zip. The files should then be installed in your Drupal installation, under sites/all/modules. You can also install the module through the Drupal Admin UI.  You should have also already installed the instrumentation, including instrumentation for your web server and PHP.

Features

Additional Layers

The module provides information on various Drupal-specific layers, each corresponding to a specific Drupal hook. Hooks are PHP functions called by the Drupal framework code at various times during request processing, and are used to generate Tracelytics layer entry and exit events. These events allow Tracelytics to determine how much time each request spent processing in an individual layer. The layers supported today are:

• drupal_hook_boot : Tasks run at the beginning of each page request.
• drupal_hook_init : Setup tasks for non-cached page requests.
• drupal_hook_form_alter : Modifications of a form before it is rendered.
• drupal_views (corresponds to time spent between drupal_hook_views_pre_view and drupal_hook_views_post_render)
• drupal_hook_node_load : Act on nodes loaded from a database.
• drupal_hook_watchdog : Message / event logging.
• drupal_hook_exit :  Tasks run at the end of each page request.

Controller / Action

A controller and action pair is reported for each request by examining currently active Drupal “menu item”, which provides information on the code handling each request. This is particularly important with Drupal since almost all non-static requests are handled by the root index.php file, meaning the URL itself is not useful for categorizing a request.

• Controller: The ‘page_callback’ function associated with the active Drupal menu item (from the menu_get_item() call).
• Action: The first argument in the ‘page_arguments’ array associated with the active menu item.

Request Partitioning

Requests are associated with a specific partition, depending on the type of request. This is crucial for determining where your heaviest requests come from, and when. The support partitions today are:

• Admin : Drupal admin user
• Anonymous : Anonymous user (not logged-in)
• Authenticated : Logged in user
• Cron : Periodic / cron execution
• Drush : Command line shell / script (see www.drush.org)

Complete Error Logging

Drupal provides a “watchdog” function for logging error/warning/debug messages. More severe messages can be forwarded to Tracelytics and will be tracked as errors.

Configuration

The above features can be turned on and off through the Drupal admin interface:

Note that it also integrates with Drush, a command line / shell scripting package.

How’s it work?

How does this module integrate with Tracelytics?  If we examine the module code, we’ll see that it makes extensive use of the oboe_log function.  This is a function added to PHP by the Tracelytics PHP instrumentation (“oboe”) module. It allows a code to log events, and is described in more detail in our knowledge base.  The particular snippet logs an “info” event to report the partition based on the currently logged in user:

switch ($user->uid) {
  case 1:
    oboe_log('info', array('Partition' => 'Admin'));
    break;

  case 0:
    oboe_log('info', array('Partition' => 'Anonymous'));
    break;

  default:
    oboe_log('info', array('Partition' => 'Authenticated'));
    break;
}

If we browse through the rest of the code (specially the tracelytics_early and tracelytics_late sub-modules), we’ll see that each module registers the same hook. The “early” module executes first and sends the “entry” event, while the “late” module executes last and sends the “exit” event. The order of Drupal module execution is controlled by assigning each module a weight.

Depending on the site configuration, various other modules would also register the same hooks and perform processing. Using the timestamps on the events allows Tracelytics to determine the processing time spent in each layer.  Snippets for the “init” hooks (which are called for all non-cached page requests) are shown below:

tracelytics_early.module:

function tracelytics_early_init() {
  if (variable_get('tracelytics_layers', FALSE)) {
    oboe_log('drupal_hook_init', 'entry', array());
  }
}

tracelytics_late.module:

function tracelytics_late_init() {
  if (variable_get('tracelytics_layers', FALSE)) {
    oboe_log('drupal_hook_init', 'exit', array());
  }
}

The pattern used for other hooks is the same.

As you can see, integrating Tracelytics as part of your application is very straightforward! In future blog articles, we’ll explore integrations with other languages and platforms, including Python, Ruby, and Java.