Listen to this podcast for a chance to win a copy of Peter’s book, Beginning Ruby!
Interviewed by Geoffrey Grosenbach
I’ll be at RailsConf in Portland, May 17 to 20, and also Ruby on Rails Amsterdam, Netherlands June 7th, and Ostrava on Rails, Czech Republic June 22 and 23. If you see me there, ask me for a PeepCode t-shirt and I’ll give you one for free, limited to supply on hand.
Geoffrey: It’s the “Ruby on Rails” podcast. I’m Geoffrey Grosenbach talking with someone many of you already know. Peter Cooper started “Feed Digest”, also “BigBold” code snippets and “Ruby Inside” blog among other things. He’s talking to us from Louth in the U.K. Did I say that right?
Peter Cooper: Yeah, you did a good job. [laughs]
Geoffrey: There are, like, 20 Peter Cooper’s in Skype in the U.K. but I was able to find you. You’re the only one in Louth.
Peter: Yeah, that sounds about right. I’ve made a little bit of a habit of getting it touch with each of them, although they seem to get in touch with me first.
Geoffrey: You know each of them?
Peter: Well, not all of them but quite a few.
Geoffrey: In the U.S., James Smith is a common name and there’s, like, a whole convention of people named James Smith.
Peter: Yeah, there was a guy trying to get all of the 4, 000 Peters to meet up in London as well, but I don’t think it ever happened.
Geoffrey: Well, most people know you from reading your blog, “Ruby Inside” and if they were to ask who was the most famous Peter Cooper they would mention you. When did you start that? It’s been quite popular, a lot of people visit it daily.
Peter: Yeah, it was kind of an unintentional serendipity, really, the whole thing. I sort of started it because an editor from Apress approached me, back in April 2006, about writing a Ruby book, which we’ll probably be talking about later.
But, they approached me about that, so I started to work on it and I thought a good way of promoting the book would be with a blog. If it got a number of subscribers, then hopefully I could count on a number of those buying the book and it would do well.
So I began “Ruby Inside” with that in mind. I would link to stuff – you know, news about Ruby and Rails. I hoped a lot of newcomers to the language would approach it because there are a few blogs around like “RedHanded” and the O’Reilly Ruby blog.
But I wanted something that was, perhaps, less kitsch and very matter-of-fact. I mean some might call it boring, but it’s just news basically. Then it ended up that all the people that do Ruby on Rails already, perhaps don’t even need to read the book, ended up subscribing to it and it’s just progressed in the last year to where we are today.
Geoffrey: That’s fascinating! So, you started the blog hoping it would become popular so that you could then use it to promote the book. That’s quite a lot of thinking ahead.
Peter: Yeah, when I started the book I could see the future and I kind of mused about the future a bit and I thought, “Well, what’s going to happen when I release this book and no one buys it?” I was kind of thinking from the start, “How am I going to sell this book?” It’s all seemed a natural way.
Geoffrey: You don’t write a lot of original articles on there, usually it’s a link and a photo of some other interesting article that’s been done. I think that’s a great idea because some other people sprinkle links in, and we definitely have a lot of tumblelogs, but nobody else was really doing that kind of thing specifically.
Peter: Yeah. It’s almost like a tumblelog in some ways. It doesn’t have the tumblelog format as such but a lot of it is, as you say, just a picture and some text.
The reason I have the pictures on most of the posts is because I saw a few blogs that were like that in totally different areas before I got going. And I thought, “When you see these pictures, you don’t look at it because of the pictures, but they keep you flowing through. It’s something interesting to see. You know, we are visual animals at the end of the day.”
So, I put that up and it seems to have worked.
Geoffrey: I think that’s a good idea. Whenever I write a blog article, I try to throw some kind of screen shot or picture in there. Especially if you’re reading through some of these aggregators – you know, “Text… text… headline… text… Oh, there’s a picture! I’ll click on it.”
Peter: Actually, none of the pictures on “Ruby Inside” are clickable. That’s something I should definitely think about for the future.
Geoffrey: I think you should. Anytime I’ve done any kind of usability testing, it’s like one of the first things – people just try to click on a big picture.
Peter: Yeah, perhaps I’ll do that from now on.
Geoffrey: Even though you’ve got a number of viewers, you don’t have any Google Ads, you haven’t even tried to sell any kind of ads on there. It’s almost like a public service or, as you said, advertising your own book eventually, when it came out. Now that the book is out, do you have any plans to try to capitalize on that or are you going to keep it ad free?
Peter: Well, one of the problems, especially in the Ruby area, is a lot of people think that this is a good thing that there’s no specific vendor that’s got a large interest in this area. You know, they haven’t got a $1, 000 tool-kit to sell. There’s no Sun or Microsoft directly in this area even though they are kind of helping on the sidelines.
It kind if limits the commercial interests in advertising. I mean, there are a few bits and pieces. There’s like, BitRails, hosting companies, small things like that. But, there’s no one who is going to spend $20 CPM, or whatever, as is common in Java sites and so on.
So I see it, at this point, as being my contribution back to the community because I don’t release an awful amount of code. I tend keep my code quite proprietary so I sort of see that as my contribution back, even if I’m not doing much Open Source stuff.
Geoffrey: I appreciate too, the frequency of posts. You’re not posting there – well it’s rarely even more than once a day. It’s manageable. A lot of these other tumblelogs are great, but when you just have that much information it’s almost overkill.
Peter: Yeah, that’s been something I’ve kept a lid on. I mean, sometimes it can slack for a couple of days, but then a lot of news will come in or there will be a lot of things to announce and then I’ll use the WordPress feature where you can post into the future to appear the next day, or the day after.
The reason for that is because I’ve read blogs where someone just wants to post every ten minutes and you end up unsubscribing from it because it’s just a nightmare. So I’ve always been careful to spread them out.
Geoffrey: Well “Ruby Inside” hasn’t been your only success. You recently sold the “BigBold” snippets site that you wrote with code snippets. Tell us about that. You started that almost just as a project to learn Rails, right?
Peter: Yeah. I began playing with Rails in November 2004. I did some tiny amounts of stuff with it, but I had other work going on so I didn’t jump into it full-time.
So in March 2005, I kind of had this need. I had all these different snippets of code that I wanted to access. And I was in love with del.icio.us at the time, in fact, still am, and I wanted something similar, where instead of putting links in, I could put actually code in. And then I could tag the code with various tags, like “Perl” or “C” or “RSS” and so on, and then put it out on a whim.
It was actually Sam Stephenson, he created a system that was similar. It wasn’t for code, as far as I recall, and I can’t remember the name of it. He never released the code. But it was a very simple content tagging system. And I saw that and I thought, “Yeah, that’s actually kind of cool. It doesn’t do the stuff that I want it to do, but I could sort of develop something that uses the same kind of concept.” Which I did.
It was basically a 24-hour project, really. It was one of these, “I’m tanked up on Red Bull, and I’m going to stay up all night long and get this done,” kind of thing.
Peter: So that’s where the first iteration of snippets came from. It was literally just for me, to start with. The growth was a very, very shallow curve, basically. But then it just did build up over time; it’s one of these viral things. And especially once I did a – holding my fingers up to do quotes now – “Web 2.0” design onto it, it really started to ramp up quick.
Geoffrey: Yeah. Early on, I remember, that was one of the first resources. When I was learning Rails, I would just search for things in Google, and inevitably, big, bold snippets would show up in a couple of the results, and so I ended up seeing a few things there and contributing. But at the first, I think it was just black and blue, and then you added some pastel colors afterwards.
Peter: Yeah. Actually, the way you found it, via Google, would have been the way that almost everyone found it, except in the latter days when I started linking to actual posts from Ruby Inside, places like that, because, just for some reason, it just did really well on Google. I mean, the PageRank wasn’t even that great, but Google just seemed to kind of respect that whole tagged hierarchy. And there’s also these pattern of links in there that made it sort of catch on quick.
Geoffrey: I found that, even searching for specific error messages – sometimes, when I have a problem, I’ll search for that exact error message, and usually, a variety of sites come up. Maybe that’ll be your next idea – I’ll just throw that out for free – is specific error messages: “Nil error on line this.” And sometimes that actually shows up often enough that it’s useful when somebody has a solution.
Peter: Yeah, I think a lot of people do that, actually. I’ve actually done that on my personal blog, where I’ve had an error message installing some gem or some library or whatever, and I’ve sort of copied and pasted the error message in, and then actually presented the solution if I have one. So yeah, people come through that way as well.
Geoffrey: Now, you ended up selling that. Somebody else approached you, and you shopped it around a bit. How did that happen?
Peter: I have a friend who owns a web hosting company, and he was interested. He’s always been interested, sort of asking me about my projects and so on. It eventually got to the point, and he started saying, “Are you interested in selling it?” It was making a certain amount of money each month. It wasn’t too far off about $1, 000 a month it was making, just off its own back. I wasn’t doing anything; that was just from AdSense.
And on the amount of traffic, that was quite a low CPM. And he’s like, “You know, I think I can do something with this, to sort of promote my own business and so on.” So I said, “Well, what would you want to pay for it?” And he gave me a figure. And I said, “Yeah, that’s a pretty good offer, actually.”
But I sort of went away, and people have told me over the years, “Never settle for the first offer. Always see if you can find anything better.” I got in touch with a few people who I thought might be interested. So I’m thinking, “Nothing’s going to happen.”
And then Rick Ross, from DZone – as I like to call it, the Digg for developers, which has become a lot more popular with Ruby developers lately – he says, “I’ll offer you X for it, as long as we just do the deal now; it’s done immediately.” And literally, within a week, the whole thing was done and cleared up.
It just went great from start to end, and they’re enjoying the traffic off the back of it. And because of the advertisers they’ve got, they can monetize it a lot better than I could, so everyone wins, as far as I see it.
Geoffrey: That’s great. So you were getting a little income from it anyway, but it was almost an incubator. And then somebody else can put the marketing, or tie it in to the rest of their site.
Peter: Yeah. I just didn’t have the network or the advertising sales team, as it were, to handle that. But he has, and now he’s got the traffic for it.
Geoffrey: Now, another one of your businesses – there are many…
Geoffrey: Is Feed Digest. It started out as RSS Digest. And I think I saw, at one time, you were getting a couple million hits to that per month. Quite a few people are using that?
Peter: Yeah. The whole thing kicked off in sort of mid-2004, where I wanted to take my del.icio.us – and everything seems to relate to del.icio.us, in some way with me, for some reason – but I wanted to take my del.icio.us links and put them in a little section at the top of my blog, so other people could see what I was bookmarking. It all started from there, because del.icio.us lets you output stuff as RSS. So processing RSS works great.
But then, as it went overtime, I used it, and then I thought, “Well, let’s let other people use it.” It grew, it grew. It was only doing 100, 000 requests a month in the first couple of months. I had to keep building it to be 10 times better than it was. So after that, I thought, “Well, I’d better build a new version that can do a million hits a month, and then 10 million,” and so on and so forth.
The system that’s in place now can handle a billion per month. It’s already about 250 million a month, digests that are actually being served, so it’s only at a quarter of capacity. And luckily, it’s at a point where there’s almost like a shared-nothing kind of architecture to it now, so I could take that one billion requests that it can do, and then buy more service and just literally spread out the load now. So I’m not too worried about the traffic now because if it gets any more beyond one billion requests, hopefully I’ll have more business-oriented things on my mind.
Peter: That’s right, yeah. That’s why the rest of it’s so high, because we have several big clients that use this on their site, like “The Denver Post,” for example. I think they are the third or fourth largest newspaper in the US – independent newspaper, at least. So, as you can imagine, their site gets several million request a months, at least, straight out of the barrel.
So yeah, it’s not like we’re serving up 250 million page views, because if we did then I would certainly be a lot wealthier than I am now. [laughs]
Geoffrey: [laughs] Now, there’s a free version of that, you can have an aggregate feed with a half-dozen different sites or something like that. And then there’s a paid version. Are you still going to have to ramp that quite a bit for that to be profitable, or is it supporting itself?
Peter: Oh, it’s definitely supporting itself. It’s just that it’s beyond me; it can’t support many other people. It’s at that kind of level, where it needs to focus more on getting the bigger clients who have specialist needs and are willing to pay a lot more money, rather than focusing on 10, 000 or 20, 000 small customers who only just pay enough to keep the thing going.
So that’s where things are going now, even to the point of open sourcing, sort of a lighter version of it that other people can then take and run themselves.
The typical sort of open source model. Where if you have a company that wants to do site complex, we already have a few customers like that, where they can then pay us a lot more money and we focus on that kind of area. Rather than the mass-market stuff. It works, but you have to be really on the ball with it. Whereas if you’ve got a few larger enterprise clients, it’s a lot easier to manage.
Geoffrey: Of course, when anyone thinks of feeds – a lot of people are going to think of FeedBurner. Is it surprising to you that a company like that hasn’t tried to do the same thing, or is it just that there’s room for many competitors in the same area?
Peter: I mean, there are a few competitors. One of the first ones that was just sort of coming into fruition as RSS Digest was beginning, was called Feedroll. They charged a lot of money, they had bits of downtime here and there. Quite a few people did migrate from them to us. There are a few competitors, but Feed Digest has definitely become the biggest because it had free accounts and the pricing structure was good value for money and so on.
But companies like FeedBurner, they really are in a completely different sort of league. FeedBurner’s kind of free service that they offer, as far as I’m concerned, is there just for mind share and good will. They make a lot of money off of their enterprise stuff, which they keep quite well hidden on their site. You sort of have to click around and find out what their enterprise services are. I’ve had friends in companies that I know have gone to them for quotes for services and the prices are kind of astronomical.
I don’t think they are going to worry too much about competing for the small fry where we are at the moment. FeedBurner is an influence in this area and that’s what’s influencing my sort of thinking for the future about moving on to more of an enterprise level.
Geoffrey: For a couple of months, I’ve been thinking, “Wow, It would be great if there was some service where you could actually plug your own ad into your RSS feed, or, I suppose, sell other ads.” Make it a WordPress plug-in or a Mephisto plug-in or even pipe it through a completely separate service. I just looked recently, and FeedBurner is going to actually offer something like that.
Do you have any plans for that for Feed Digest or is it going to stay funded by the people who are using it, rather than advertising?
Peter: I considered advertising, but again, you need quite a few people on staff to handle that kind of stuff. You also need the contacts; there’s a lot of networking to do. It’s definitely not a technological problem. It’s very much a sort of cultural, person problem, when you get selling advertising. Especially, people are going to complain if we forced it on free users without going through a process. Or even launching a new brand to do that.
But what you actually said about putting ads in your feed. Obviously, Feed Burner do have the ads that they can put in. But if you wanted to put your own ads in, I think the company Text Link Ads, that you might be familiar with, I think they offer a WordPress plug-in you can use. They’ll set ads up for you, but you can also put your own adverts into your own feed. You can actually turn it off so that they can’t sell when you do your own, so it might be worth looking at that.
Geoffrey: OK. So somebody’s thought of that already.
Geoffrey: Well, technically I’m sure a lot of people listening to this have been frothing at the mouth when they heard that you’re doing two billion requests per month. Do you have any kind of tips? Are you using Rails for part of that? Are you using file-based caching? What kind of things did you have to do to make that work?
Peter: All right, now, just to correct: It’s 250 million.
Peter: So a little bit smaller, but…
Geoffrey: But capable…
Peter: Yeah. One of the sad things that I should admit, is that because of the age of our RSS feeds [indecipherable] on Feed Digest, most of the back-end technology is in Perl. Which I know you’ll sympathize with. What’s been happening, over the course of that time, is that stuff has been migrating to Ruby. The front-end has always been Ruby on Rails because everything goes through a database, so Rails is the perfect thing for the front-end. The back-end, that actually serves the Digest, does the millions of requests per month, still runs on Perl.
But in the last couple of months, I’ve moved the entire sort of feed-crawling system, the system that reads in the feeds and then puts the list of data entries into the database, that was moved entirely over to Ruby now. The funny thing about that is, the Ruby feed crawler is about 20 times faster than the Perl one. A lot of people have been kind of scratching their head thinking about how that happened. I don’t know whether it’s because I’m a good Ruby developer or just a very bad Perl developer.
I think it has a lot to do with how the XML is being parsed. I’m sort of going into Ruby geekery here. The Perl system worked with an Expat library, which actually should be fast, but the way I was processing it, it wasn’t. But now the Ruby parser uses why the lucky stiff’s Hpricot. However you pronounce it. I always call it aitch-pricot. It uses that mainly because a lot of feeds out there are invalid; bad XML, this, that, and the other. Using the Expat library just becomes a nightmare when you’re dealing with invalid XML. It will complain all the time and so on.
Whereas if you’re using Hpricot, which isn’t really meant for parsing XML, or dealing with the structure, you can actually parse in an extremely liberal way. Hats off to why the lucky stiff for that one. It’s improved the parsing speed no end.
Just to sort of finish off your last question about what the caching’s like. It is coming from Perl but it’s using MemCache d, which most RSS people would be familiar with now. And there is some file caching as a backup to that so if the MemCache server dies for whatever reason – and it hasn’t in two years – but if it dies, it will fall over to file-caching on the fly and things like that. So it’s very basic caching kind of stuff. You know, trying to hit the database as little as possible and all that kind of thing. So nothing to out of the ordinary.
Geoffrey: So, in two years, the MemCache server has never – I’m sure you’ve had to restart it at times – but it has never died on its own?
Peter: I think the longest uptime on it was something like 369 days or something. I think that was the longest uptime for the whole thing, without a restart. I think that was for my SQL server as well, somewhere in that area. I focus more on securing the machines rather than upgrading the software all the time. Although that does happen in fits and bursts.
Geoffrey: Impressive. I’m surprised nobody has written a book on a lot of that. Of course, once you start to use it, it’s not that difficult, but you know, MemCache, MogileFS, some of those other things that they developed at LiveJournal. There aren’t many books out there telling you how to use that.
Peter: No, that’s true, actually. And even the documentation is very basic. I think Brad Fitzpatrick is a little bit like me. If he developed something, he either developed it for his own use, or he – there’s not too much documentation there. You’ve got to figure out a little bit for yourself. So no, there are not any books on that.
Even the whole deployment situation; knowing how to hook things up like Mongrel and Rails and MemCache, and administering a Unix server of some description. You sort of have to pick bits and pieces up from all over the place. Blog [indecipherable] has been an excellent source for that, but there is no great book that helps you with the whole Unix web applications or Web 2.0 deployment issue at any level, really, that I know of.
Geoffrey: Speaking of books, you wrote your book. “Beginning Ruby” for Apress just came out only a couple of weeks ago. To start with, when I saw that I was like, well, you know, a lot of people are going to recommend the “pickaxe book” that’s been out for a couple years’, and I first learned Ruby from the Poignant Guide, Why’s “Poignant Guide to Ruby”, who wrote the intro for you, the forward. But why did you feel like there was a need for another book on learning Ruby?
Peter Cooper: I kind of used my own experience as the basis for it. The whole idea for the book came from the publisher initially. This could sound a bit like making up a reason for writing a book after the deed is done, but there was definitely a lot of my own experience in there. My experience was that I got into Rails before I got into Ruby, which I think a lot of people share. I thought, ‘I’m a Perl developer, I don’t really want to learn another language, I’m happy with Perl, I’ve been doing eight years or whatever it was at the time. I don’t want to move.’
So I did buy the books like the Pickaxe, and I saw the Poignant Guide. I mean I read the Poignant and I could sense the great skill and the great humor in it, but I didn’t connect to it on an academic level. You have to be of more of a creative mindset straight of the bat to get into that. While I enjoyed it and looked at drawings, and thought Why was a standup guy, I didn’t connect to it for learning Ruby, it didn’t appeal to me. So I thought ‘Well, I’ll buy the Pickaxe guide, it’s a more technical option, just the plain facts, plainly put.’. So I bought the Pickaxe, and it’s a good book, it helped me a lot in the reference side of things.
But in the actual [indecipherable] and the instructional side of things, it tended to assume quite a lot. That you were familiar with dynamic languages and object-orientation and so on. And obviously coming from Perl, where object-orientation is just an absolute mess, apologies to Larry wall, but it’s just such a mess that some of the concepts have different names, or you’ve never bothered to use them because they look completely arcane. So I think the Pickaxe did assume quite a bit of knowledge before coming into things.
But then Chris Pine’s book “Learn How to Program”, which is also a great Ruby book but perhaps starts at too low a level. So I wanted something that is the sort of book that is the sort of book that I read when I was younger, in the 80’s, I read these books like “Learn Basic” and so on. They had a sort of the way they went through things, ‘that you do this, this is a variable, look what happens when you had these variables, look what happens when you do print a+b’ that kind of thing. I wanted to take that approach, take the ‘80s programming book standard, and bring it to the Ruby side of things.
So I hope that is where it struck, hope that it will become a book that someone who is a C developer, or someone who is playing Excel macros or something like that, they have an idea of what programming is. They can install programs and they are happy with using the command line, perhaps. But they don’t know how to actually develop programs in Ruby, then, it’s for those guys.
Geoffrey: Well I was impressed, I thought you did a good job. Even though it goes through a lot of detail it’s still possible to read it quickly. There’s a lot of content, 600 pages or so, but I sat through the last couple nights and read through 100 pages or something like that. It definitely covers at a nice pace for someone who is trying to get up to speed with Ruby but already knows a little bit about programming.
Peter Cooper: One of the things about the book is that like a lot of the 80s programming books that I read when I was younger, it does make some assumptions, and there are certainly some style in there that are unique to me, as such; but, there is one or two ways of doing things, whereas in Ruby there are perhaps ten or twenty ways of doing things, I’ve chosen to focus on a certain way of moving through.
Now, I mean the reason for that is a: because of the amount of space, but b: because the main goal of the book is someone can read, they know how to develop Ruby applications, but they are just given enough and told there’s a few more ways of doing, that they then become interested in signing up for the Ruby mailing list, actually getting on the IRC channels, actually posting stuff to their blogs and so on. Actually becoming a part of the community, actually go out and look for stuff sort of in the way that I did as I learned Ruby.
Geoffrey: To me the real accomplishment of that book is getting Why’s lucky stiff to do an illustrated introduction, three or four pages, how in the world did you arrange that?
Peter Cooper: Yeah, that’s been kind of interesting. I mean he’s an interesting guy to get a hold of, sometimes you don’t hear back from him, sometimes you hear back from him straight away. Definitely and interesting guy. I mean and I and definitely all the people at A press appreciate the job he’s done. It’s just come out great. The first time I read it I was laughing already, it just looked so good. I really like the spin he’s put on it about the stuff about chopping animals and putting them back together. It’s only Why that could look at it that way.
Literally, he was my first choice for the person to do the forward. People will think ‘yeah, lets get Mats to do it’, no disrespect to Mats, but I wanted something to soften the blow for the start of the book. It was something a little bit jovial that people could look at before diving into the more serious stuff. I thought Why was the perfect person for this, but I thought there was no way he would agree to this. But it was literally, an email and he came back and said ‘Yeah, I’d love to do it’. There was no stress to it at all.
Geoffrey: I’m [indecipherable] to be a little documentation of history, I think that’s only the second time he’s been in print, at least in the mainstream press. One of the chapters of the Poignant guide was in one of Gerald Spolsky’s anthologies, but now your book is going to show up in libraries and schools across the country, and has a little bit of Why’s lucky stiff illustration in it.
Peter Cooper: Oh definitely, I definitely want his genius to spread. Because even though I didn’t connect immediately with the Poignant guide, I love it now. Its stuff like that the other languages don’t have. there is no equivalent for pi-form [indecipherable] to capture those creative people that wouldn’t have come to Ruby or Rails without that sense of fun in the community.
Geoffrey: Well, this chat has gone quickly, I’m sure that there is a lot else that we could talk about, thanks to your publisher, I am going to run a little contest for somebody who wants to win a copy of Peter’s book “Beginning Ruby”. I won’t give too many details, but the trick is going to be if you can come up with a Google search that returns the text of the book Oliver Twist as the first result, but you cannot use the words ‘Oliver’, ‘Charles’, or ‘Dickens’. If you can figure that out, send that to email@example.com with Peter Cooper in the subject and first three people A press will send you a copy of Peter’s book. For the rest of you, you can go to your favorite online store or bookstore and get a copy. Well thanks Peter. Are we going to see you at any events coming up, or are we going to have to go to Louth to see you?
Peter Cooper: Yeah, people are always welcome here. I haven’t got many events coming up, but there are some very interesting new announcements that people should look out for from Ruby Inside this week.
Geoffrey: Do you want to give us any hints or we’ll just have to stay tuned?
Peter Cooper: No, afraid not, you’ll just have to stay tuned.
Geoffrey: No hints. Well thanks a lot. It’s been great to chat with you. We’ll keep up with you at rubyinside.com and petercooper.co.uk.