i am dustin diaz

About that slowness on Twitter...

One #newtwitter feature we love is "infinite scroll"- we fetch more tweets in the background well before you can reach the bottom of a timeline, so that you can keep reading without the added friction of clicking a button and waiting. It looks something like:

$(window).bind('scroll', function () {
 if (nearBottomOfPage()) {
   // load more tweets ...
 }
});

Last week we noticed that after a couple of these background fetches, scrolling would slow to a crawl. After some investigation, we found that the culprit was a single jQuery selector:

$details.find(".details-pane-outer");

This allows you to scope a find() within the context of a root node, and is standard practice in jQuery. Aside from upgrading from jQuery 1.4.2 to 1.4.4, no code had changed. We didn't catch the issue because no immediate problem was found and the application continued to function normally according to our Selenium and QUnit tests.

We reverted the upgrade, and #newtwitter returned to its regular snappy state, effectively allowing us to isolate the issue to the latest jQuery release. Our hypothesis is that something changed in the way context selector searches work. If the un-scoped selector $('#some-id') is used, the slowness issue goes away in 1.4.4. Of course, our code is filled with context selector searches (with good reason), so we found it best to simply downgrade to jQuery 1.4.2 for now.

The jQuery team has been made aware of the situation, and we can hopefully work together to identify and test a fix to this performance regression. Cheers!

notes: Mark Christian ( @shinypb ) helped discover this issue which eventually lead us to investigate further into jQuery selector calls. Also, Patrick Ewing ( @hoverbird ) helped edit my initial 3rd grade writing skills.

[UPDATE]: Just to speak to a few really great questions in the comments.

this jquery bug doesnt sound like the ground reason for this particular perf issue but rather a sample case of slow browsers being affected by events that fire excessively like scroll

It was easily isolated to jQuery. Literally changing from 1.4.4 to 1.4.2 the change was significantly noticeable without even benchmarking (in each respective browser). It would be like answering the question "is a slug slow?"

It seems more likely to me that the problem here is not .find(). If you’re really comparing performance a class-based selector vs. that of an id-based one

You'd think. We actually tried a find() based on an 'id' just for kicks. Eg: $node.find('#thing'); (although you'd never really do that ;) — it was still unbearably slow. Our hypothesis is that something changed in jQuerys contains() method, because regardless of doing an id selector, if you're doing it from a find(), it MUST check to see if it's contained within the context provided.

Was there a noticable difference if you used $(‘div.details-pane-outer’)?

The context selector is required for our case (you could have multiple details panes in memory). But if your question is regarding adding the 'div' to the selector — it made no difference (although you would think it would).

I'm not a JS developer, but isn't it a best practice to write a function that checks for some timeout before executing the heavy-duty stuff?

Well, you're right in that you don't need to be a JS Developer to know that you should probably re-evaluate expensive code being run at high intervals. We've been experimenting with creating a lazyBind for high frequency events that throttle the callbacks despite there still actually being a bug in the upgrade.

Why do you not just cache the results in a variable?

Good call. We're doing that in several other areas of the app, it's a shame we missed that here.

Where you guys able to create a simplified testcase? Seems like that would be step one in trying to figure out what the problem REALLY is.

We did isolate it to that method. I believe the jQuery team might be doing some more investigation.

[UPDATE 2] John Resig has posted an update to his blog.