Archive for the 'Development' Category

Temporarily closing free project registration

November 2, 2008

Hey everybody!

I know things have been pretty quiet. Man, it’s tough running a service like this on your own. Don’t worry though, things should start ramping up soon with all the things set in motion behind the scenes.

It’s funny too though because DevjaVu is getting more free signups than ever. It’s funny because free project registration requires an invite code. I know we have a public invite code that anybody can use, but it still means people are going out of their way to search for an invite code.

Unfortunately, we’re dealing with some growing pains and we’re getting a lot of heavy load on our servers from Subversion usage. We’ve got an interesting plan to solve this (which I shouldn’t talk about yet), but for now and because our registration process has been somewhat flaky, we’re going to turn off the invite code for free projects in the next 48 hours. This means you won’t be able to signup for new projects until our big re-release.

Now about that… basically we’re repositioning the DevjaVu service and so it’s going to be renamed and the pricing structure will change a bit. It’s also getting a brand spanking new look both on the site and the default project theme. And if things go as planned, we’ll be releasing with a completely open free project plan.

So yes, after a period of stagnation we have changes abound coming up. But for now, we’re closing free project registration within the next 48 hours. Sorry guys!

-Jeff

Ugliness averted

December 7, 2007

That’s better. Not perfect, but nothing will look as good in IE. At least I got to brush up on the timeless art of cross-browser styling.

Actually I think I cheated because I just used conditional styles for IE and IE7. Anyway, this has been a nice reminder that we still have to think about IE (in fact, in two flavors now). I’m really happy to get back to non-browser issues now.

Hope you guys are enjoying the free accounts and spreading the word. They won’t last long!

-Jeff

Consider daylight saving time with cron

November 8, 2007

I know it’s over a week after the fact, but a friend said it’s probably worth mentioning our experience with this recent daylight saving time switch. Ignore this if you don’t observe DST or your servers don’t use it (which might not be a bad idea).

If you have cron scripts that are meant to run only once a day and will run on Sundays (such as daily or weekly schedules), do not run them on or between 1am and 2am. Otherwise you risk them running twice or not at all.

Yes, it may seem obvious, but it’s probably overlooked. We certainly didn’t realize it, but since we just happened to be debugging a weekly cron script running at 1am that Sunday, we caught it in the act of doubling a long running, CPU intensive task.

Happy Wednesday!
-Jeff

Service testing and alerts with Twitter

October 27, 2007

Have you ever messed with those site alert services that ping your site and email you if the site is down? For free services, they can be pretty handy. We used Montastic, made by the same guys that built Mojo Helpdesk, which we still use. The problem is that they don’t tell you much else, like if your registration page is broken or an SVN operation fails. Plus it would be nice to get IM or SMS alerts instead of emails.

So with a little work we took Mechanize (also available in Python and Perl) and a test framework and made high-level tests of our service’s functionality. For Rails users, it’s a bit like making integration tests, only it runs on the actual site, so it doubles as a more thorough site pinger. The downside is that you don’t really have fixtures, it’s all real data and real interactions. That can make it a bit tricky, but if you can get around that, you’ll have a nice service testing script you can run locally or on another server.

You might want to just put it in the other server’s crontab, but unless you get emailed the cron output, just running the tests is pointless without some kind of alerting. Even if you get cron output, you probably don’t want to see the test results every time it runs. If we have to do our own alerting, we might as well get fancy…

It turns out getting fancy is not all that hard thanks to Twitter. With almost no work at all you can write a wrapper for your tests that grabs the test status line (the one with the dots) and the name of the test that fails and then send them as a direct message to you with Twitter’s super simple web API. Then you get messages like these sent to your IM or even to your phone if you have direct messages enabled for it:

...F...... registration (Registration is broken)

......FF.. svn checkout (SVN checkout is broken)

FFFFFFFFFF homepage (The whole site is down)

Once you have your test script done, you can use a simplified version of our Ruby wrapper script to have your own super awesome free IM and SMS alert system:

require 'net/http'
require 'uri'

username = (your twitter user)
password = (your twitter pass)

alert = (array of twitter users to alert)

ruby_bin = (full path to your ruby, since run by cron)
test_path = (full path to your tests)

url = "http://#{username}:#{password}@twitter.com/direct_messages/new.xml"

output = `#{ruby_bin} #{test_path} 2>&1`
test_stats = output.split("\n")[2]
failure = output.match(/\d\) ((Failure|Error):\n([^\(]+))\(/)
if failure
 failure = failure[3].split('_')
 failure.shift
 failure = failure.join(' ')
 alert.each do |u|
  Net::HTTP.post_form(URI.parse(url),
   {'user'=>u, 'text'=>"#{test_stats} #{failure}"})
 end
end

See? Simple! Now you can get woken up in the middle of the night as soon as something breaks…

-Jeff

DevjaVu Web API

October 23, 2007

Recently we enabled XML-RPC access to your Trac projects. Not only does this mean you can use the advanced features of external Trac interfaces such as Mylyn for Eclipse (previously known as Mylar), but of course that you can have programmatic access to your tickets, wiki, search and milestones.

You can see all the exposed functions by going to /login/xmlrpc on your project and authenticating via HTTP if necessary. This is the URL you would use to give other programs access to your Trac, including your own. Here are some examples of using the API with Python.

We really value having lots of inputs and outputs as a web service, so we’re glad to finally make this API available. We also have web hooks that allow you to trigger your own scripts when commits happen. Just check out the Hooks section of your Admin.

Over time, we look forward to exposing more via API and hooks. One possibility is to expose SVN operations via API, so that you can programmatically use and control your repository. Imagine powering a website using the DevjaVu wiki as a content management system and exposing your repository to users via an easy to use web interface that accepts and versions the contents of zip files.

We love API’s because they open up the possibility to build great things easily. We hope you can come up with some creative uses of our API. Let us know if there’s anything else in particular you’d like exposed!

-Jeff

Fun with tracd clusters

June 2, 2007

Actually it wasn’t all that fun. Thanks for putting up with the random Trac outages. We’ve reverted back to CGI based Trac to keep things stable again until we can make sure tracd clusters work without dying all the time. We would have reverted back sooner, but we needed enough information in the production environment to debug back in development.

To explain this in full technical detail, we have Adrian Perez, an up and coming writer of children stories:

Once upon a time, I had some fish. They used Trac, but they all died. Then there was a picture of some fish, eating insects and larva. But they died to. It was a fanciful time for all, said the Fox. I knew that would happen said the Dog. I couldn’t tell however, as I was precipitously on the shitter. Sorry kids. It was only a mistake. Before that though there was obviously an outage to say the least. I popped it. Brace with the hips when using Subversion. I could tell you about the events of this tremulous time, but if I were to, Ninjas, as always, would attack. Most of the server outages are actually Ninja based or related. In the time of Pirates, I tried to count them all, but they were legion.

Thanks for being such awesome beta users!

-Jeff

Better SSL coming

May 2, 2007

Since we released SSL access, we’ve been running into certain limitations with the way we configured it. Some of you may have noticed, for example, you were unable to use svn move over secure SVN. Our new setup should fix this, however for the time being, SSL will be disabled for Trac until we can redesign our whole configuration for the new SSL configuration.

-Jeff

Notifications and Akismet

April 18, 2007

Tonight we deployed email notifications for Trac tickets, so you and your team can stay on top of changes. You can enable notifications in the Notifications panel of the Admin area. You can also customize the Reply-To address and turn on various “always notify” options. Subversion email notifications are coming soon.

We also enabled the Akismet spam filtering strategy, so if any of you have public projects and want to add extra spam protection, you can put in your API key and take advantage of Akismet’s great service. We already provide several builtin spam filtering strategies that seem to do a really good job, but now you can make it even stronger. If you don’t have an Akismet API key, you can sign up at wordpress.com and get a free personal key.

With all this anti-spam tech, we’d really hate it if any of you are getting spam. If you encounter any kind of spam problems, please notify us so we can tweak our system.

Also, you may have noticed the menus have changed a bit. We’ve simplified the meta navigation links and put them in the top right where our DevjaVu community links used to be. We put those under the Admin area. In the near future, we’re going to give you a filterable headlines feed of our blog in there, so you can be sure to stay up to date on new features, possible downtime, important announcements, etc.

Anyway, I hope you enjoy the new features! Tell us what you think!

-Jeff

Normalizing user emails

April 10, 2007

We just normalized our user database and forms so that all emails are compared as lowercase. That means that Trac permissions and authentication will be case insensitive.

This makes things much easier and a bit less confusing. However, we were not able to get Apache’s authentication for Subversion to authenticate while ignoring case, which means for now you’ll be required to use lowercase emails when authenticating for Subversion. For most people this should be a non-issue since most emails aren’t mixed case, but for the rest of you, we hope this doesn’t inconvenience you too much. We may be asking you to break habit, but at the same time it’s less keys to press. ; )

If it becomes that big of an issue, we can start hacking Apache modules. But we’d rather focus on more important issues in this infrastructure phase, like migrating all your data to a replicated PostgreSQL database and getting Trac off CGI onto tracd clusters.

-Jeff

Getting serious

November 20, 2006

In about a week we’ll be deploying what’s basically a completely rewritten DevjaVu based on Trac 0.10! Along with all the greatness of Trac 0.10, this update is going to give you automatic spam filtering, commit message triggers, and wiki enhancements. Most importantly though, it should open the floodgate to more long awaited features, fixes, and of course, our premium packages.

You see, what we have live now is really just mature prototype code. For the most part it gets the job done, but without things like a proper vendor branch, it’s almost impossible to upgrade Trac. That’s the reason we’ve been on Trac 0.9 for so long.

This newly engineered version uses Rake for the build process of setting up and installing Trac and its plugins. I have got to say that Rake is great. We also use Rake to automate a lot of our system management processes. I just automated our deployment process with Capistrano, which makes it all the more easier and quicker for us to get out new features and fixes. If you have the means, I highly recommend trying Rake and Capistrano for even non-Ruby projects.

Anyway, see you in about a week with the new release and some more exciting announcements. Our traffic keeps climbing, invitaton pleas keep rolling in, and the feedback has been great. The best part is we haven’t even gotten started yet…

-Jeff