Over the years I've used a variety of Linux Distros: Mandrake, Red Hat, FreeBSD, Fedora, Gentoo, and Ubuntu. Distrowatch keeps track of everything we need to know about the distros, and recently there has been an enormous push in desktop Linux thanks to Dell putting Ubuntu on desktops and Compiz-Fusion bringing snazzy eye candy to even low end machines. Distrowatch gives some pretty decent stats on the main Distros but for a while I've wanted to know how Google sees their popularity; mainly by how many pages mention specific distros.
Using some python, a spreadsheet, and a little scraping, I was able to get my answer. To see how Google would rank different distros I'm using the number of results Google returns when searching for the Distro's name as my numbers. I'm going to write a HOWTO on the technical aspects of what I did sometime this week, but here are the basic steps
In a Google Spreadsheet I made a sheet that held the names of the top distros on Distrowatch.
Another sheet holds the full list from distro watch (366 on record at the time of this writing)
I set up a dapp to take these names, and return the number of results Google would have if you searched them
A python script pulls the distros out of the spreadsheet, queries the dapp, and puts the results back into another sheet
I have 2 sets of results. One is a query using the vanilla list out of the spreadsheet. The second is appending the word Linux to the distro if it does not already have it as the title, I was curious as to how this would effect the results. Below are the results of the most popular Distros on Distrowatch. Look, Ubuntu! The spreadsheet that has all of the findings (and all 366 distros) is shared here
Distro
Page Hist
Ubuntu
96,800,000
FreeBSD
36,700,000
Fedora
35,800,000
openSuse
29,500,000
Debian Linux
28,100,000
KNOPPIX
12,500,000
Mandriva Linux
5,720,000
Gentoo Linux
4,430,000
PCLinuxOS
3,170,000
Slackware Linux
3,010,000
MEPIS Linux
1,640,000
Stay tuned for the code behind it! Subscribe to the feed to get more updates.
A few days ago I posted some Overheard in New York Google Posts. Today is some Overhead in the Office. It's not quite as good as the orriginal, but the office posts are still really funny. If you like these, subscribe to the feed, enjoy!
No Problem, Then
Coworker #1 on speakerphone: How do I Google something?
Coworker #2: Go to W-W-W dot Google dot com. Then just use it like any other search engine.
Coworker #1: What's a search engine?
Coworker #2: You know, like when you use Yahoo or MSN to look something up.
Coworker #1: I've never Googled before in my life, and I never want to again!
Overheard by: it actually got worse
Or Time Management. Whatever.
Office manager: I'm going to Google time travel!
What Happens When You Use Google China
Office manager: Civil War Battlefield Protection, how can I help you?
Woman on phone: Yes, I'm trying to find information on the Civil War and I just can't.
Office manager: Well, have you tried the internet?
Woman: Yes, I typed "Civil War" into Google and NOTHING comes up on the Civil War.
Office manager: ...Really?
Woman: Yes... so can you tell me who was in the Civil War?
Office manager: That would be the northern states and the southern states.
Woman: Not the British?
Office manager: Um, that was the Revolutionary War.
Overheard by: Astonished
Just Scan in His Photo and Use Face-Recognition Google
Boss: I went to school with this guy...I don't remember his name, but he just became the president of some real estate development company in DC. Can you look him up on the internet? I'd like to send him a note.
I Think You'll Like the Results
Boss : What is that red thing on top of a rooster's head called? I can't find a description anywhere on the Internet.
Employee: Just Google 'cock' and 'diagram.'
Internship
Boss: How many servers do you think Google has?
Lackey: Infinite.
Boss: Infinite? You're a retard.
Overheard by: choking on a brownie
Technical Know-How
Boss on phone: Wait, wait, wait, does your computer have Google on it? Yeah, just type it in there.
Update PR File
CTO: My interview article is in USA Today!
Programer: Can you send me the link? Never mind, I'll just Google "USA Today"
Back to Work
Engineer: I'm against Google Earth! The terrorists are using it! And the communists!
Sensitivity Training
Boss: So what's the going rate for hiring midgets these days?
Boss: Yeah, I'm sure you can just fire up Google and type in "Midget to hire tri-state" and somthin's gonna pop up.
Send Out Press Release
Boss: So see if you can find these people's email addresses.
Intern: ...You want me to find Desmond Tutu's email address?
Boss: Try Google if you get stumped.
Back to Work
Producer: My friend went to the Galapagos Islands and was astounded. They have birds called blue boobies. Google "blue boobies". You'll see pictures of them.
Suit: I'm not searching for blue boobies on my computer. I'll get called into the office for a talk.
Producer: Oh, I'll do it...see?
Suit: Wow, who would have thought that would't have brought up a porn site?
Scan PC
Tech: Oh crap, I've been infected by Google.
Install Yahoo!
Boss: Do we have Google installed on our internet?
IT guy: We put it on your machine yesterday.
First, Techcrunch came up with a list of the top blogs that users subscribe to using Google Reader. The top 3 on the list were BBC with 202,463, Google News wih 192,100, and ESPN with 189,274 . Remember, these are only the stats if the subscriber is using Google Reader, obviously there are blogs with a higher subscription count, such as the Official Google Blog (They still haven't moved over to blogs.google.com eh?) with over 669,000 subscribers as reported by FeedBurner.
It was all well and good that Google Reader showed high stats for these blogs, but.... we were apparently duped...
In his article Google Reader Stats are BullSh*t (With Proof), Pete Cashmore goes into his explanation of the "default feed effect." A default feed is a feed that comes already subscribed in a news aggregator , or is part of a default group (like say Digg, Engadget, and Gizmodo being in the Technology group on Google Reader). Being in these groups is a major traffic boost.
However the system has some flaws
There is subscription data for feeds that don't exist
Google Reader does not check to see if a subscriber is active or not. (Someone signs up for Reader, adds a feed, and never touches Reader again, they count as a subscriber)
Feedburner pulls some stats from Google Reader
This problem is not limited to Google Reader. It includes many feed readers and start pages
Tune in next time after I do some research into how out wack FeedBurner statistics are.
Over the coming days/weeks I'll be updating the template on the rest of the domain to coincide more with the blog, as well as adding the top menu to the blog. The blog is still powered by blogger and the old address nicely forwards to the new one, posts included. I also updated the FeedBurner feed so if you use that to subscribe you shouldn't have to change anything (although with the forwarding I don't think it matters, but why risk it). I also set up gpowered.net and www.gpowered.net to forward to the blog as the main page. Any of my python examples will be moved from http://gpowered.net to http://gpowered.net/g/. I'm going to update all of the posts to reflect this.
Why it won: Mappington is the most inclusive map / location information hub I have ever come across. "Mappington is a user created atlas of the world." It covers the following:
Maps: Google Maps, Virtual Earth, Yahoo Maps, Google Earth
Photos: Flickr
Videos: Youtube
Wikipedia Entries
Opinions / Rating section
Social Web on Wikipedia, Facebook, del.icio.us, furl, Yahoo MyWeb, Wink, ma.gnolia, spurl
Discussions
History
News: Yahoo News
Blog: Google Blog Search, Technorati
Hotels
Flights
Weather: Weatherbug
Events: e:vene
Trips: Yahoo Travel
The layout is crisp, clean, and easy on the eyes. There are minimal load times and Mappington makes good use of Ajax. If you are planning a trip or just want to explore new places, Mappington is the best place to do it. I'm definitely going to use it to plan out my trip to Ireland in a few months.
What I think it could improve on:
The video's don't show that you can click on them (mouse pointer does not change)
The photos should tap into Picasa Web also
In the News section, it looked like there was supposed to be a left frame (Google News maybe?) but it never loaded
I wish you could blow the map up full screen
The thumbnail view in Google Maps wasn't working
Mappington should build some widgets / gadgets to get more exposure on personal web pages and sites like myspace
API's: Amazon eCommerce, CNET, del.icio.us, Digg, eBay, Flickr, Google Search, Windows Live Search, Yahoo Search, YouTube
The search box doesn't understand spaces?!? Ex: I put in "Tim Broder" it returned nothing and asked me if I meant "Tim+Broder" which returned some results that I expected
CADoogle
Demonstrates Autodesk Freewheel. Allows you to search, preview, and download publicly available CAD files indexed by Google. It is free to use and leverages Google Search and Autodesk Freewheel Web services.
If you haven't heard about Overheard in New York ( OHNY ), or don't subscribe to it, I can't recommend it more. It's absolutely hilarious. It's been around for a while and is constantly updated with a dozen or so quotes a day that people hear while walking / riding around New York City. You don't have to live in the city to enjoy it, though you might not get all of the humor. I used Google Reader Search to go through the feed looking for Google references. Here are a few, note: there were a lot more but not all were appropriate for all audiences.
He's Started Scooping Matt Drudge
Female Google suit: How's your son doing?
Male Google suit: Great! He's 11 months now, and he's starting to get a personality -- it's great!
Female Google suit: I'd love to see some pictures sometime!
Male Google suit: He's got a blog! --Elevator, Port Authority building, 15th & 9th
But Which One? Which One?
Crazy guy: Can anyone tell me about the Chinese stock market? [Two minutes later] Can anyone tell me about the Chinese stock market? Come on, people, that's an easy question!
Suit, laughing: Here's an easy question for you: How far away is Saturn to Pluto?
Crazy guy: That is too easy. Why would you ask me something so easy? The answer is Google. Google is the answer. The real question is, are there aliens from Saturn and Pluto on this train right now? And how could you tell the difference? [Looks around, waits for an answer.] Well, I know, because I'm one of them! [Crazy guy gets off the train.] --1 train
But We'll Find Out for Sure When the Anime Comes Out
Nerd #1: So, was she hot?
Nerd #2: According to Google Image search, yes. --Math Building, NYU
She Has Six Months Left to LiveJournal
Doctor guy: Okay, we have the chest x-ray and it explains what's going on. See this lesion? It is pretty impressive.
Patient lady: "Impresssive"? Is that a medical term? Am I gonna have to google that, too? --Office, 71st & Park Wednsday One-Liners
Sketchy doctor: I Google everything! I treat my patients with Google. --Mt. Sinai School of Medicine, 98th & Madison
Teen chick on cell: Yeah, it's funny... Google it. Wait, do you guys even have Google in Florida? --54th & Park
Girl: Google is, like, totally taking over the world! --Terminal 4, JFK
My first ticket for the Google Document List API has been fixed. As I posted about before, the Document List API did not have the ability to just retrieve the documents from a given folder or tag. This has now been made possible, thanks API team!
Original ticket:
It would be ideal if we could pull back a list of documents from a certain
folder. This would give more functionality and make the size of the data
smaller if you only needed to grab a list of file from a certain folder.
Possibly something like
gdata.docs.service.DocumentQuery(folder=['myfolder1'])
or
gdata.docs.service.DocumentQuery(folders=['myfolder1','folder2'])
Most people are familiar with Picasa's usual features; Editing, organizing, uploading, printing, etc, but Mike Herf, the Picasa Engineering manager is going to take us on a tour on some of the "easter egg-ish" features. He's going to be posting hidden features and tips on Wednesdays over at the Google Photos blog so check it out.
The first up is search. While you might be familiar with searching for a filename, or a directory you know you put the photo in but can't find, you might not be aware that you can also put in dates, camera models, flash (for if one was used), what type of lens was used, or even the iso settings. All of these are indexed by Picasa and will be returned in a search. Another fun search term is color:red or color:gray. Try is out ;) The last hidden search feature is under Tools->Experimental->Show Duplicate files and Picasa will list all the files that show up more then once on your hard drive, pretty nifty for cleaning space.
For more features on Wednesdays, subscribe to the photo blog.
Google has been using it's blogger platform to deliver news about it's products and services to the masses. Including the multi-language blogs, there are about 76 Google blogs out there. But how often do they post? I went through each one (of the English blogs) and figured out the average number of posts, per month, that they have for 2007 so far. If a blog wasn't around back in January, I took that into account also.
The top two are the official Google Blog, and the Google Code blog, which makes sense as they are all encompassing aspects of Google. Following these are Maps and Adsense, two of Google biggest products.
This chart is live from the spreadsheet I made which you can view here
So I've decided to add a new feature to gPowered, the Google Mashup of the week. So many of these get created all the time using various Google APIs and services. Each week I'm going to search around find a dozen or so of the newest ones, pick out the one I like the best and why... with a possible runner up. As for when I'll be doing these, the new mashups will be posted sometime in between Saturday and Monday. Usually on the weekend unless I'm going away... like next weekend (Tennessee).... and the weekend after that (Boston)...
... and the Mashup of the Week goes to:
Jolocate
APIs: Google Maps, Craigslist
The Good: Its craigslist... with a map. Or I should say its a map with Craiglist in it. If you've ever gone apartment hunting, you know the value of this. Smooth, no lag when changing grids, Craiglist opens right in the same window. I also really like the account feature. The site tries to profile you for people looking for roommates, and also from what I gather, is going to email me suggestions when I need to start looking for a place for a July 1 move-in.
The Bad: Not necessarily bad, but I would have liked the option to have Craigslist open in a new window
The Ugly: New York wasn't in the default list!
Other contestants:
Spell Boy Beta
APIs: Google Gadgets
Why it didn't make it: For on the web FireFox or Google toolbar can spell check all the forms for you, and if you're mail client can't spellcheck... well.. get a new one.
http://www.careersix.com/
APIs: Google Maps, Indeed
Why it didn't make it: The website was VERY slow to load the first time, I almost stopped looking at it because I thought it was down. Also I didn't see any place to search by job? only by location and then I have to hit ctrl+F?
Checker-forum.de
Small community that uses the Youtube API to show off and comment on videos in users profiles.
APIs: Youtube
Why it didn't make it: I don't speak German
100 Mile Diet
APIs: Google Maps
"When the average North American sits down to eat, each ingredient has typically traveled at least 1,500 miles. Use this interactive map and the 100-Mile Diet book to eat local, think global."
FormSpring + Google Calendar
APIs: Google Calendar, formspring
Why it didn't make it: Formspring costs money for anything worthwhile, free gives u 3 forms and 50 saves per form?
The Picasa team has gotten a blog in the form of the Google Photos Blog. I think this brings the number of Google blogs to 75 (including the international blogs). On the photo blog they will be posting feature updates to Picasa, photography tips, and the team's favorite photo albums.
I've been wanting to share what I subscribe to in Google Reader and using the functions I wrote I was able to do just that. Check out the article for the full run down on the unofficial Google Reader API. This is written in python but should be easily portable to php. If i get around to it, I want to make a WordPress plugin so bloggers can share what they read with their readers. This will be followed (or in parallel depending on my mood) with a Javascript version so Blogspot users can do the same in the sidebar. On to the code!
To start off we'll just copy the functions we need from last time. Generally this is the login and SID token functions, as well as the feed list function.
from django.shortcuts import render_to_response
from django.template import Library
from elementtree import ElementTree
import urllib
import urllib2
import re
login = 'timothy.broder@gmail.com'
password = '***'
source = 'gPowered'
google_url = 'http://www.google.com'
reader_url = google_url + '/reader'
login_url = 'https://www.google.com/accounts/ClientLogin'
token_url = reader_url + '/api/0/token'
subscription_list_url = reader_url + '/api/0/subscription/list'
#login / get SED
def get_SID():
header = {'User-agent' : source}
post_data = urllib.urlencode({ 'Email': login, 'Passwd': password, 'service': 'reader', 'source': source, 'continue': google_url, })
request = urllib2.Request(login_url, post_data, header)
try :
f = urllib2.urlopen( request )
result = f.read()
except:
print 'Error logging in'
return re.search('SID=(\S*)', result).group(1)
#get results from url
def get_results(SID, url):
header = {'User-agent' : source}
header['Cookie']='Name=SID;SID=%s;Domain=.google.com;Path=/;Expires=160000000000' % SID
print url
request = urllib2.Request(url, None, header)
try :
f = urllib2.urlopen( request )
result = f.read()
except:
print 'Error getting data from %s' % url
return result
#get a specific feed. It works for any feed, subscribed or not
def get_feed(SID, url):
return get_results(SID, get_feed_url + url.encode('utf-8'))
#get a token, this is needed for modifying to reader
def get_token(SID):
return get_results(SID, token_url)
#get a list of the users subscribed feeds
def get_subscription_list(SID):
return get_results(SID, subscription_list_url)
Then we'll want to get rid off all the information in the feed that we don't want and load what we do into a data dictionary. After its in the dictionary, feed names and links (and the folders they are in) are ready to be displayed. As usual, I use Django to display my pages, but everything is the same up to the final return in the Feeds method. Below is an example of what each subscription looks like in the Google Reader Feed, and below that is how to process it
class myFeed:
def __init__(self, name, link):
self.name = name
self.link = link
def Feeds(request):
SID = get_SID()
feeds = get_subscription_list(SID)
tree = ElementTree.fromstring(feeds)
d = dict()
#loop through each feed
for object in tree.findall('list')[0].findall('object'):
strings = object.findall('string')
key = object.findall('list')[0].findall('object')[0].findall('string')[1].text
#tag already exists, add to the list
try:
d[key].append(myFeed(strings[1].text, strings[0].text.replace('feed/', '')))
#tag doesn't exist, create list
except KeyError:
d[key] = [myFeed(strings[1].text, strings[0].text.replace('feed/', ''))]
return render_to_response('pages/feeds.html', {
'feeds': d,
})
For those of you that use django or are just curious how I end up displaying the feeds, this is what i have in my view:
One of my friends at work asked me today how I sift through volumes of news to find articles and information on Google and specifically on Google Code for my articles. The answer is simple, I don't. Granted, I DO read a LOT, but there just aren't enough hours in the day to read everything I want to. So, I have a Yahoo Pipe sift through the feeds for me. Pipes is "a powerful composition tool to aggregate, manipulate, and mashup content from around the web." Its fairly easy to use and doesn't require any code or coding experience, although understanding the basic layout of an RSS or Atom feed is a definite plus. The method I'm going to describe could work for any blogger or anyone who wants to filter a single or large group of feeds by keyword. I'm not going to go into the details of Yahoo Pipes, there are plenty of tutorials and examples for that. I'm just going to describe how I built the small pipe I use to filter for Google news.
The first step is to log into or create a Yahoo account and go to http://pipes.yahoo.com/pipes/ and click on Create Pipe. The first node we're going to use is under sources and is called "Fetch Feed". Add as many of these as you want to pull in all the feeds you will need.
Then use unions (located under Operators) to join these Fetches together. If you have more then 5 feeds, you will need multiple unions.
Finally, add a filter, also located under Operators. Here, "Permit" items that match "any" of the following. Then, add rules where item.description > Contains and then the word or phrase you want to search for. This filter will allow any posts that have the words through, and block everything else. Send the output of the Filter to the Pipe Output and you are done.
You can then subscribe to the output of this pipe with the reader of your choice. When I subscribe to my pipe's feed in Google Reader, I only see posts that contain the word Google or google. I also could have done that as one filter with a regular expression, or just with "oogle".
ProBlogger's Birthday is here and Darren is running small contests throughout the week with over $54,000 in prizes. I missed the first few, thank you work and being on call, but the prizes have been pretty good so far.
The first contest was for copies of Aaron Wall’s SEO Book and all you had to do was comment on the post announcing the contest. The second set of prizes were free listings on SeekZap and all you had to do was sign up! (This is the one I'm really bummed about missing.) The final prize for yesterday for the first 25 responders was tubes of Mederma scar treatment. For the full prize list, see here
You don't have to have a blog to participate in most of the competitions so if you want to have some fun over the next few days, and into the weekend (this is actually pretty good for me, I'm stuck home on call, now I'll have something to do ;), add the feed to your reader and try to keep up.
Its definitely been a busy month for Google Apps with the launch of presentations (finally). As a result, there is a new video as well as an updated tour. There is no API available yet but as soon as it comes out I'll post a HOWTO.
Spreadsheets added new ways to pull external data. The external sources can include XML, a URL, HTML, or any feed based file. Spreadsheets also added autofill (type Dec and it can autofill December. Type Jan, Feb, Mar, and it will fill in the rest of the months for you. This also works with days of the week, time, and more)
The Reporting API was release into Google Apps Premier, Education, and Partner editions. This allows users to see detailed reports on how people in their organization are using Google Apps. They can track account activity, disk space, email client usage, and more.
Or at least to London? A VW bug with a camera attached to the top has been spotted in London. Could be Google? Maybe expanding to areas around their offices in other countries? Nothing has been confirmed yet.
I think this could pick up if Google started going to major Global Cities. Think about it, you could go site seeing without ever leaving your desk. The top cities on my list would be London, Tokyo, Paris, Venice, Dublin, and Hong Kong. I am hopefully going to two of these in the next 6 months (Tokyo and London) around the holidays. Although, Google might have some legal hoops to jump through if they decide to do this.
