Limit Google Search Results by Date

Tech Recipes goes into how how to restrict Google search results by date: Appending "&as_qdr=d" to a search will display a drop down that lets you choose how far back in time to search.

A Greasemonkey script will also do this for you. Pretty useful if you are searching about a brand new issue. This would have helped me a lot when I was having a problem with Hardy

Fixing Nvidia 8600 GT on Hardy Heron

After updating to Hardy I got a white screen of death upon rebooting. After some research it turned out that this was more of an Nvidia driver issue rather then Ubuntu. This may not have been the most correct way to fix it, but it worked for me.

1. Completely remove nvidia-glx-new
   
   
2. Remove nvidia-kernel-common (this also removes linux-restricted-modules), (I am not sure if step 2 is required)
   
   
3. install build-essentials
   
   
4. Download the Nvidia beta driver that came out on April 10. You can get it here
   
   
5. hit ctrl + alt + f1 to break out of gnome
   
   
6. "sudo /etc/init.d/gdm stop" to shutdown the X server
   
   
7. sudo sh NVIDIA-Linux-x86-173.08-pkg1.run to install the driver. If it asks you to update you're xorg.conf file, let it.
   
   
8. Reboot. ("sudo reboot") You should now be able to enable Desktop effects. YAY FISH!
   
   

This issue has also been addressed on the Ubuntu Forums and on launchpad

Note: I have a Dell XPS M1530

Upgrade Ubuntu to 8.04 Hardy Heron

Hardy Heron came out this morning and ubuntu.com is very much down for the count

You can Still update though, click here for the Google Cache version.

Happy Updating!

New Features in this release:

Install on an existing filesystem without overwriting /home
When I moved from feisty to gutsy, I decided to do a fresh install. One of the things I had to do was back up my home folder, and when I finished installing gutsy I just copied it back onto my computer. This new feature will allow people to install the new version of Ubuntu without it overwriting their home folder.I'm sure this will come in very handy for people who like trying out different distro's.

Hardy Hardware Detection
This is more of a bug fix than a new feature. Gutsy already has excellent hardware support and the plan for hardy is even better and more robust detection of hardware. Sounds good to me.

GDM Face Browser
One of the changes I made to my gutsy is the GDM. I replaced the old one with something that allows me to just click on a picture of my username and login. This will hopefully be the default for hardy.

Auto Detection of Monitor Frequency
While I was testing gutsy beta I had to manually configure xorg.conf to get it to the right resolution. It wasn't fun. This should be a thing of the past with hardy as it will automatically detect everything for you. Huzzah!

Apt Authentication Reliability
Have you ever had an update fail for no reason? Well it actually fails because of 'transient network failures'. The aim is to make hardy more robust against these errors.

Redesign Restricted-Manager Code
They want to expand the role of the restricted manager and change it so that other distro's can share the joy.

Handling Full Disks
I've never had this problem with Ubuntu, but if your disk gets full, things can get quite ugly. They plan to add a notification and disk clean-up tool when your running low on space.

Desktop Effects
Make compiz fusion more robust and easier to use.

New Theme
Hardy Heron will be getting a shiny new theme, I hope they move away from the brown theme and choose something lighter and more fresh.

Easy File Sharing
To allow people to easily share files over a network. Not more I can say about this.

Dual/Multi Monitor Support
Currently you have to manually tweak Ubuntu if you want to use more than one monitor. They want to fix this for hardy.

Integrate Prefetch into Ubuntu
I noticed a slightly increased start up time in gutsy compared to feisty. Hardy will use file prefetch and other optimisations to speed up boot time.

Automatix-Ubuntu Team Collaboration
Automatix was extremely helpful for me in feisty. Although I don't use it in gutsy, its good that they are collaborating with the automatix team.

Single Click Install
Installing software is already pretty straightforward in Ubuntu. They want to make it even easier to install third party applications. I'm not complaining.

Apparmor Integration
This is already a part of gutsy, the plan is to increase integration to make Ubuntu even safer.

Firewall
Make it easier for users to configure their firewall.

Third Party Apt
Now when you install third party apps, you have to manually add the software repository to the sources.list. This spec makes it easy for users to install third party software and have it update automatically.

Revamped Logout Screen
They want to streamline the options you have when you click that big red button, to make things less confusing.

Better Integrated Wine
Better Wine will make it easier for Windows users to convert, thus helping to solve bug #1.

Xorg 7.3
This is one of the features that missed the gutsy deadline. This should make manual configuration of xorg.conf obsolete. Another much anticipated feature is Bullet Proof X, which will go into a graphical safe mode if anything goes wrong with X.

Slick Boot
To improve the boot and shutdown process and also make the things look nicer.

Displaying what you read from Google Reader

I've been wanting to share what I subscribe to in Google Reader and using the functions I wrote I was able to do just that. Check out the article for the full run down on the unofficial Google Reader API. This is written in python but should be easily portable to php. If i get around to it, I want to make a WordPress plugin so bloggers can share what they read with their readers. This will be followed (or in parallel depending on my mood) with a Javascript version so Blogspot users can do the same in the sidebar. On to the code!

To start off we'll just copy the functions we need from last time. Generally this is the login and SID token functions, as well as the feed list function.

from django.shortcuts import render_to_response
from django.template import Library
from elementtree import ElementTree   
import urllib
import urllib2
import re

login = 'timothy.broder@gmail.com'
password = '***'
source = 'gPowered'

google_url = 'http://www.google.com'
reader_url = google_url + '/reader'
login_url = 'https://www.google.com/accounts/ClientLogin'
token_url = reader_url + '/api/0/token'
subscription_list_url = reader_url + '/api/0/subscription/list'

#login / get SED
def get_SID():
    header = {'User-agent' : source}
    post_data = urllib.urlencode({ 'Email': login, 'Passwd': password, 'service': 'reader', 'source': source, 'continue': google_url, })
    request = urllib2.Request(login_url, post_data, header)
   
    try :
        f = urllib2.urlopen( request )
        result = f.read()
   
    except:
        print 'Error logging in'
       
    return re.search('SID=(\S*)', result).group(1)

#get results from url
def get_results(SID, url):
    header = {'User-agent' : source}
    header['Cookie']='Name=SID;SID=%s;Domain=.google.com;Path=/;Expires=160000000000' % SID
    print url
    request = urllib2.Request(url, None, header)
   
    try :
        f = urllib2.urlopen( request )
        result = f.read()
   
    except:
        print 'Error getting data from %s' % url
   
    return result

#get a specific feed.  It works for any feed, subscribed or not
def get_feed(SID, url):
    return get_results(SID, get_feed_url + url.encode('utf-8'))
   
#get a token, this is needed for modifying to reader
def get_token(SID):
    return get_results(SID, token_url)

#get a list of the users subscribed feeds
def get_subscription_list(SID):
    return get_results(SID, subscription_list_url)

Then we'll want to get rid off all the information in the feed that we don't want and load what we do into a data dictionary. After its in the dictionary, feed names and links (and the folders they are in) are ready to be displayed. As usual, I use Django to display my pages, but everything is the same up to the final return in the Feeds method. Below is an example of what each subscription looks like in the Google Reader Feed, and below that is how to process it

class myFeed:
    def __init__(self, name, link):
        self.name = name
        self.link = link

def Feeds(request):
    SID = get_SID()
    feeds = get_subscription_list(SID)
    tree = ElementTree.fromstring(feeds)
    d = dict()   

    #loop through each feed   
    for object in tree.findall('list')[0].findall('object'):
        strings = object.findall('string')
        key = object.findall('list')[0].findall('object')[0].findall('string')[1].text
       
        #tag already exists, add to the list
        try:
            d[key].append(myFeed(strings[1].text, strings[0].text.replace('feed/', '')))
        #tag doesn't exist, create list
        except KeyError:
            d[key] = [myFeed(strings[1].text, strings[0].text.replace('feed/', ''))]

       
    return render_to_response('pages/feeds.html', {
    'feeds': d,
    })

For those of you that use django or are just curious how I end up displaying the feeds, this is what i have in my view:

My Reading

{% for item in feeds.items %}
{{ item.0 }}
    

        {% for feed in item.1 %}
        
{{ feed.name }}
        {% endfor %}   
    
{% endfor %}

Again, too see what I subscribe to, click here

HOWTO: YUI Tabview

A few days ago I added the Digg counts to the bottom of the Posts page on gPowered.net. Although, the more posts that I add, the further down on the page this section will get, so I decided to play around with YUI's tabview control and put the post list in one tab, and the diggs in another. It turned out to be really easy:

First we need a few dependencies

<!-- Dependencies -->  
<!-- core CSS -->  
<link rel="stylesheet" type="text/css" href="http://yui.yahooapis.com/2.3.0/build/tabview/assets/tabview.css">   

<!-- optional skin for border tabs -->  
<link rel="stylesheet" type="text/css" href="http://yui.yahooapis.com/2.3.0/build/tabview/assets/border_tabs.css">   

<script type="text/javascript" src="http://yui.yahooapis.com/2.3.0/build/yahoo-dom-event/yahoo-dom-event.js"></script>  
<script type="text/javascript" src="http://yui.yahooapis.com/2.3.0/build/element/element-beta-min.js"></script>  

<!-- Source file -->  
<script type="text/javascript" src="http://yui.yahooapis.com/2.3.0/build/tabview/tabview-min.js"></script>  

And then we just need to organize some DIVs

<div id="demo" class="yui-navset">   
 <ul class="yui-nav">   
  <li class="selected"><a href="#posts"><em>Posts</em></a></li>  
  <li><a href="#diggs"><em>Diggs</em></a></li> 
  <li><a href="#HOWTOs"><em>HOWTO's</em></a></li> 
 </ul>               
 <div class="yui-content">   
  <div id="posts" style="margin: 10px">
   <h3>Posts</h3>
   content
  </div>
  <div id="diggs" style="margin: 10px"> 
   <h3>Diggs</h3>
   content
  </div>
 </div>
</div>

HOWTO: Getting the Numer of Diggs from Digg (Python)

After 2 of my posts were on the Digg front page this morning (Thank you all very much to those that dugg them), I took my first look into the Digg API. I wanted a way to take a quick look to see how many Diggs certain stories were getting. In some ways it is similar to GData: make a call to a URL, get some XML back, parse it, etc. It does, however, feel lighter, probably due to its streamlined nature. It has one purpose, get information off of Digg. Using this, I've added a section in the Post List section of gPowered.net that shows the Diggs of a few of the articles that I have submitted on Digg.

The API is broken into 5 main sections or endpoints. Each of these will return related types of data:
- Stories
- Events
- Users
- Topics
- Errors

In this quick HOWTO I'm going to take a quick look into the Stories endpoint so I can display the number of Diggs specific stories have. We'll start off by making a small class to hold our returned data (useful to send to a template or just for working with later on. We don't want to keep having to hit the ElementTree to get data out). All of the calls will be send to 'http://services.digg.com/'. In this example I will only be querying 'http://services.digg.com/story/{story clean title}'.

import httplib2  
from elementtree import ElementTree  

#for storing
class MyDigg:
 def __init__(self, title, link, digg, diggs):
  self.title = title
  self.link = link
  self.digg = digg
  self.diggs = diggs
 
 def __str__(self):
  return self.title + ' ' + self.diggs

#stories to get diggs of  
posts = [
 'Google_NOT_releasing_it_s_Goobuntu_Desktop_OS_STOP_DIGGING_IT', 
 'New_Digg_Home_Page_breaks_the_Linux_section_on_IE',
 'Google_Reader_API_Functions'
 ]
 
#hold returned info
my_diggs = []

#all calls go through this
digg_service = 'http://services.digg.com/'

#just looking at stories
service_endpoint = digg_service + 'story/%s'

#only need 1 result back
trailer = '?count=1&appkey=http%3A%2F%2Fgpowered.blogspot.com'

#keep track of total diggs
total_diggs = 0

After we are set up, we will want to loop through each story we want to get Digg data for. Add the well formed title into the query string, and send it to the Digg service. Then, parse the response, and get the information we need.

for story in posts:
 curr_story = service_endpoint % story
 url = curr_story + trailer

 h = httplib2.Http() 
 resp, content = h.request(url, "GET", body="nt", headers={'content-type':'text/plain'} )
 
 story = ElementTree.fromstring(content).findall('story')[0]
 
 d = MyDigg(story.findall('title')[0].text, story.get('link'), story.get('href'), story.get('diggs'))
 total_diggs = total_diggs + int(d.diggs)
 my_diggs.append(d)
 print d

print 'Total: ' + str(total_diggs)

And that's that. my_diggs now has all the information we need!

HOWTO: Google Reader API Functions

I've been wanting an API for Google reader since I started using it, and especially since i started gPowered so I could display a list of the feeds I read on the site. The official word on an API for reader is "It's coming in a few weeks," but that was back in late 2005. The reason being that at the time, the URLs the API would use were going to change a lot. So, after a bit of research and coding I came up with some python functions to do the job.



The first step was authenticating against Google accounts without using the client library. The Python Gdata Library makes login very easy but Reader isn't part of the Client Library yet (maybe I'll try to add it, we'll see...) but this was the method I was using for gdata and python pre-Client Library, and the principles still hold true for working with Reader. Thankfully, most of the research for working with the 'Reader API' was done for me already by Niall Kennedy. This is an unofficial, unsupported API and the URLs for some of the queries have changed since the writing of that article. Here we go...

We're going to use urllib(2) to handle the communication with this one. I rather would have used httplib, but I was having trouble with the authentication cookie. Each retrieval has its own URL to query against

import urllib
import urllib2
import re

login = 'timothy.broder@gmail.com'
password = '****'
source = 'gPowered'

google_url = 'http://www.google.com'
reader_url = google_url + '/reader'
login_url = 'https://www.google.com/accounts/ClientLogin'
token_url = reader_url + '/api/0/token'
subscription_list_url = reader_url + '/api/0/subscription/list'
reading_url = reader_url + '/atom/user/-/state/com.google/reading-list'
read_items_url = reader_url + '/atom/user/-/state/com.google/read'
reading_tag_url = reader_url + '/atom/user/-/label/%s'
starred_url = reader_url + '/atom/user/-/state/com.google/starred'
subscription_url = reader_url + '/api/0/subscription/edit'
get_feed_url = reader_url + '/atom/feed/'

When we authenticate against Google Reader with a gmail account and password in the browser, a cookie is stored. We'll have to recreate the values in this cookie. The static values are the Domain (.google.com), the Path (/), and Expires (we'll use 160000000000). The unique value, based on the current login session, is the SID (Session ID?), which we will need to retrieve. We'll do the login and retrieval in the same function:

#login / get SED
def get_SID():
    header = {'User-agent' : source}
    post_data = urllib.urlencode({ 'Email': login, 'Passwd': password, 'service': 'reader', 'source': source, 'continue': google_url, })
    request = urllib2.Request(login_url, post_data, header)
    
    try :
        f = urllib2.urlopen( request )
        result = f.read()
    
    except:
        print 'Error logging in'
        
    return re.search('SID=(\S*)', result).group(1)

We'll also need a function that can handle any of those URLs, create the header, attach a cookie to it, and retrieve the data from Google. I left the return as a raw data string so you could use whatever XML parsing library you want. I personally like using ElementTree.

#get results from url
def get_results(SID, url):
    header = {'User-agent' : source}
    header['Cookie']='Name=SID;SID=%s;Domain=.google.com;Path=/;Expires=160000000000' % SID

    request = urllib2.Request(url, None, header)
    
    try :
        f = urllib2.urlopen( request )
        result = f.read()
    
    except:
        print 'Error getting data from %s' % url
    
    return result

The following methods are the calls that I've gotten working so far; I'm going to keep working on the 'edit' functions, like adding, removing feeds, changing tags, etc. See the comments for what they do. Note: Any edit against the API needs to send over a changing token as part of the call

#get a token, this is needed for modifying to reader
def get_token(SID):
    return get_results(SID, token_url)

#get a specific feed.  It works for any feed, subscribed or not
def get_feed(SID, url):
 return get_results(SID, get_feed_url + url.encode('utf-8'))

#get a list of the users subscribed feeds
def get_subscription_list(SID):
    return get_results(SID, subscription_list_url)

#get a feed of the users unread items    
def get_reading_list(SID):
    return get_results(SID, reading_url)

#get a feed of the users read items    
def get_read_items(SID):
    return get_results(SID, read_items_url)
    
#get a feed of the users unread items of a given tag    
def get_reading_tag_list(SID, tag):
        tagged_url = reading_tag_url % tag
        return get_results(SID, tagged_url.encode('utf-8'))
        
#get a feed of a users starred items/feeds
def get_starred(SID):
    return get_results(SID, starred_url)

#subscribe of unsubscribe to a feed    
def modify_subscription(SID, what, do):
    url = subscription_url + '?client=client:%s&ac=%s&s=%s&token=%s' % ( login, do.encode('utf-8'), 'feed%2F' + what.encode('utf-8'), get_token(SID) )
    print url
    return get_results(SID, url)
    
#subscribe to a feed
def subscribe_to(SID, url):
    return modify_subscription(SID, url, 'subscribe')

#unsubscribe to a feed
def unsubscribe_from(SID, url):
    return modify_subscription(SID, url, 'unsubscribe')

Example usage:

SID = get_SID()
print get_subscription_list(SID)
#print get_reading_list(SID)
#print get_read_items(SID)
#print get_reading_tag_list(SID, 'me')
#print get_reading_tag_list(SID, 'nada-mas')
#print get_starred(SID)
#print get_token(SID)

#test_feed = 'http://picasaweb.google.com/data/feed/base/user/timothy.broder/albumid/5101347429735335089?kind=photo&alt=rss&hl=en_US'

#print subscribe_to(SID, test_feed)
#returns ok but I don't see the feed in reader?

#print get_feed(SID, test_feed)

Like I said, I'd like to keep going with this and get the edit functionality to work better. I'm also going to take a look into the Client Library and see if I could set this up as a patch that people could use if they wanted to use the API.

View Google Groups posts in Reader

I don't know why I never noticed this before but you can subscribe to the mail sent to Google Groups through an RSS feed at the bottom of each group. I'm trying it out for a few groups, I might like it better then reading through the mail. I wonder if there is a way to subscribe to the rolled up versions of the posts...

HOWTO: Pulling Google Bookmarks with Python

I love using Google Bookmarks (usually with the Google Toolbar) because it lets me get to my bookmarks at home on my laptop or desktop, at work, or anywhere. It's great. Now I'm using those bookmarks to power the links section of gPowered.net
First we're going to need the httplib2 library so we can authenticate against Google and grab the bookmark feed and then the ElementTree to help process the rss feed.

import httplib2
from elementtree import ElementTree

Then we'll setup the link to pull the rss from, authenticate against the request, and pull back the feed

login = "timothy.broder@gmail.com"  
password = "*****"
url = 'https://www.google.com/bookmarks/?output=rss&num=1000'  
  
h = httplib2.Http()  
h.add_credentials(login, password)  
 
resp, content = h.request(url, "POST", body="nt", headers={'content-type':'text/plain'} )

I figured for this a hashmap (or dictionary) would work well using the tags on the bookmarks as keys, pointing to lists of bookmarks. Then when we display them, just iterate through the keys. I also kept a list of the keys to make sorting faster later on. So we define our objects and then loop through the rss object pulling out the tags for keys, the names of the links, and the urls. I define a small Bookmark class which holds a name and url which will go into the hashmap to make storing the bookmark's easier. When I try to add a bookmark to the dict's list I try to append it, if the key(tag) doesn't exist I know I have to start a new list.

class Bookmark:
 def __init__(self, name, link):
  self.name = name
  self.link = link
  
d = dict()
sort_keys = []
for item in tree.findall('item'):
 key = item.findtext('{http://www.google.com/searchhistory}bkmk_label')
 if (key != None) and (key != 'gpowered') and (key != 'BP') and (key != 'Quick Searches') and (key != 'Me'):
  title = item.findtext('title')
  link = item.findtext('link')
  try:
   d[key].append(Bookmark(title.encode('utf-8'), link))
  except KeyError:
   d[key] = [Bookmark(title.encode('utf-8'), link)]
   sort_keys.append(key)

Then we'll sort the key list and the list of each key. To do this we need a small function that defines how to sort a bookmark

def bookmark_compare(a, b):
 return cmp(a.name, b.name)

sort_keys.sort() 
for key in sort_keys:
 d[key].sort(bookmark_compare)

Check out the static HTML version Here. I also made a fancier version using YUI's TreeView

HOWTO: Getting a list of post titles from blogger (Python)

This will be a quick one on how to pull the titles from your blog. I'm using it to Lists the posts I have available on gPowered.net. Firstly we'll set up our imports and call to the blogger service.

from elementtree import ElementTree 
from gdata import service
import gdata
import atom
import getopt
import sys

blog_id = 413573351281770670
blogger_service = service.GDataService('timothy.broder@gmail.com', '*****')
blogger_service.source = 'Blogger_Python_Sample-1.0'
blogger_service.service = 'blogger'
blogger_service.server = 'www.blogger.com'
blogger_service.ProgrammaticLogin()

For this query we're going to use the summary feed because all we really need for this is the titles, not the full posts:

query = service.Query()
query.feed = '/feeds/' + str(blog_id) + '/posts/summary'
feed = blogger_service.Get(query.ToUri())

Then I just do a little counting so I can use the links on my site. All the information we need is in feed.entry

curr_id = int(feed.total_results.text)
for entry in feed.entry:
 entry.my_id = curr_id
 curr_id -= 1

Quick Docs Api Example (python)

To use the gdata docs python client you need to upgrade to 1.0.7 or higher. First thing is to import the modules you'll need.

import gdata.docs.service
import gdata.docs

Then, set up the usual authentication parameters for the client.

gd_client = gdata.docs.service.DocsService()
gd_client.email = 'timothy.broder'
gd_client.password = '*****'
gd_client.source = 'gpowered-docs-list-ex'
gd_client.ProgrammaticLogin()

The most basic query will just return all of your documents

feed = gd_client.GetDocumentListFeed()

However, if we want to display just the spreadsheets, we build the query like this:

q = gdata.docs.service.DocumentQuery(categories=['spreadsheet'])
feed = gd_client.Query(q.ToUri())

Finally, we output the titles

if(len(feed.entry) == 0):
        print 'No entries in feed.\n'
for i, entry in enumerate(feed.entry):
        print '%s %s (%s)' % (i+1, entry.title.text.encode('UTF-8'))

If we wanted to, we could also import the DateTime library and show when the document was last updated

import gdata.docs.service
import gdata.docs

from mx import DateTime

gd_client = gdata.docs.service.DocsService()
gd_client.email = 'timothy.broder'
gd_client.password = '*****'
gd_client.source = 'gpowered-docs-list-ex'
gd_client.ProgrammaticLogin()

q = gdata.docs.service.DocumentQuery(categories=['spreadsheet'])
#feed = gd_client.GetDocumentListFeed()
feed = gd_client.Query(q.ToUri())

if(len(feed.entry) == 0):
        print 'No entries in feed.\n'
for i, entry in enumerate(feed.entry):
        dt = DateTime.ISO.ParseDateTimeUTC(entry.updated.text)
        print '%s %s (%s)' % (i+1, entry.title.text.encode('UTF-8'), dt.strftime('%m/%d/%Y %I:%M %p'))

For me this outputs:

1 TDP2006 Contact Info (11/18/2006 05:41 AM)
2 contact info (07/23/2006 08:15 PM)
3 Tim and Rob (08/09/2007 10:18 PM)
4 nyc happy hour spreadsheet (07/04/2007 08:25 PM)
5 public_spring_2006_roster (10/16/2006 12:40 AM)
6 dax2006 (11/12/2006 11:23 PM)
7 project dream (07/13/2007 03:54 AM)
8 Stuff Tim should get (06/13/2007 01:53 AM)
9 Erg Test Results - 9/26 (10/15/2006 01:02 AM)
10 Head of the Charles Regatta Itineary (10/17/2006 04:54 PM)
11 tvshows (11/02/2006 11:44 PM)
12 HF (10/01/2006 03:36 PM)

HOWTO: Displaying Blogger feeds with PHP

This HOWTO is going to follow the basic structure of the Python one.

To start out you'll have to grab the Zend Google data Client Library and then set the include_path so you can use it

ini_set("include_path", ".:../:./include:../include:/home/gpowered/webapps/php/includes/ZendGdata-1.0.1/library");

We then import the parts the we'll need:

require_once 'Zend/Loader.php';
Zend_Loader::loadClass('Zend_Gdata');
Zend_Loader::loadClass('Zend_Gdata_Query'); 
Zend_Loader::loadClass('Zend_Gdata_ClientLogin');

One of the first things we're going to have to do is authenticate with google services. There are two ways to do this: AuthSub proxy authentication which has a user login using their own credentials, and ClientLogin username/password authentication where you send a username and password. We will be using ClientLogin. I built a small class called gPoweredBlogger to hold the different parts I will need for this example.

class gPoweredBlogger{
 private $user;// = 'timothy.broder';
 private $pass;// = '**************';
 private $service;// = 'blogger';
 
 private $blog_id;// = '413573351281770670';
 private $uri;// = "http://www.blogger.com/feeds/" . $blog_id . "/posts/default"; 
 private $show_num;// = 5;
 
 private $client;// = Zend_Gdata_ClientLogin::getHttpClient($user, $pass, $service);
 private $gdClient;// = new Zend_Gdata($client);
 private $query;// = new Zend_Gdata_Query($uri);
 
 private $total_posts;
 
 public $output;

Then we start setting up our call to the service.

 public function __construct($user, $pass, $blog_id){
  $this->user = $user;
  $this->pass = $pass;
  $this->service = 'blogger';
  
  $this->blog_id = $blog_id;
  $this->uri = "http://www.blogger.com/feeds/" . $this->blog_id . "/posts/default"; 
  $this->show_num = 5;
  
  $this->client = Zend_Gdata_ClientLogin::getHttpClient($this->user, $this->pass, $this->service);
  $this->gdClient = new Zend_Gdata($this->client);
  $this->query = new Zend_Gdata_Query($this->uri);
  $this->total_posts = $this->get_total($this->query);
 }

For more info see the blogger developer's guide with php or the Google Account Authentication documentation
After we have authenticated with Google we need to start building up our query to GData. The first thing you'll need is your blog's id.
You can use the function in the dev guide to help you with this if you don't already know it.
Like the Python version, the below function returns the total number of posts that are in the feed. We can get a small response by sending 0 for the max results. Below is the function and the small response we get from it.

 private function get_total($query){
  //query for no posts
  $this->query->setParam('max-results', '0');
  $this->query->setParam('start-index', '1');
  
  //get back entryless feed 
  $feed = $this->gdClient->getFeed($this->query);
  return $feed->totalResults->text;
 }

 5
 0
 1
 Blogger
 Tim
 tag:blogger.com,1999:blog-413573351281770670
 
 
 
 
 gPowered
 2007-07-18T10:55:06.728-05:00

So we get the total number of posts and then we can start pulling data. Lets make a generic function, PostFrom, that can be used to show multiple posts, or just single ones, depending on what you pass to it. The start number that is passed to PostFrom has been set to the first post in the blog is considered to have an id of 1 and the latest post is the same as total_posts. This is useful so if viewers want to bookmark the page they are looking at, the post that is being displayed will not change. The following are the different functions that will make use of it.

 //show latest posts
 public function Posts(){
  return $this->ListPosts($this->total_posts);
 }
 
 //show posts starting from a certain point 
 public function ListPosts($start){
  $start = $this->total_posts - $start + 1;
  return $this->PostFrom($start, $this->show_num);
 }
 
 //show a single post
 public function Post($start){
  $start = $this->total_posts - $start + 1;
  return PostFrom($start, 1);
 }
 
 //show count number of posts starting from a certain point
 private function PostFrom($start, $count){
  //query for count number of posts starting at the given post
  $this->query->setParam('max-results', $count);
  $this->query->setParam('start-index', $start);
  $feed = $this->gdClient->getFeed($this->query);

Now we have all the data we need in the feed variable. Its been turned into an object so we don't have to worry about XML parsing here. Every node has become an objects and lists. Objects for single nodes(title), and lists for where there are multiple nodes of the same name (entry, link)

  //for links
  $curr_id = $this->total_posts - $start + 1;
  
  //normalize data for output
  foreach($feed->entries as $entry){
   //id for links
   $entry->my_id = $curr_id;
   $curr_id -= 1;

  }

Of course we're going to need next and previous buttons as well. The way we've set up the math with total_posts and the start number, we only have to increment or decrement these by count (the number of posts to display on a page). I also set part of the link, as well as the page title, that I will use below in the HTML.

  $prev = $this->total_posts - ($start - $count) + 1;
  if($prev > $total_posts){
   $prev = null;
  }
   
   
  $next = $this->total_posts - ($start + $count) + 1;
  if($next < 1){
   $next = null;
  }
  
  //showing single post 
  if(count == 1){
   $link = 'post';
   $title = $feed->entries[0]->title->text;
  //listing posts
  }
  else{
   $link = 'posts';
   $title = 'home';
  }

The final part is to make a quick object that we can use in the HTML to output everything

  $this->output = new Output($feed->entries, $title, $prev, $next, $link);
 }
} 
 
class Output{
 public $entries;
 public $title;
 public $prev;
 public $next;
 public $link;
 public function __construct($entries, $title, $prev, $next, $link){
  $this->entries=$entries;
  $this->title=$title;
  $this->prev=$prev;
  $this->next=$next;
  $this->link=$link;
 }
}

To the HTML!

The first part consists of displaying the post itself, along with its relevant information. So lets built up our objects

$blog = new gPoweredBlogger('timothy.broder', '*************', '413573351281770670');
$blog->Posts();

$output = $blog->output;

Below all the php we can run through out output object and display the posts

<? foreach($output->entries as $entry){ ?>
     <h2><a href="/post/<? echo $entry->my_id ?>"><? echo $entry->title->text ?></h2></a>
     <? echo $entry->content->text;
     $datetime = strtotime(substr($entry->published, 0, 10) . ' ' . substr($entry->published, 11, 8 ));
     ?>
     <p>Posted by <? echo $entry->author[0]->name->text ?> on <? echo date("m/d/Y",$datetime) ?> at <? echo date("g:i a",$datetime) ?></p>
     
      <div id="divider"></div>
      <?}?>

That's all for now. A working example is here

First Google Gadget

After being inspired about Google Gadgets from the Google Developer Podcast I came up with on that my old crew team can use. We have a shared google calendar that some of us use to keep track of races, meetings, etc. This gadget pulls and formats it nicely for the google IG or desktop.



Javascript code for the gadget:

<style type="text/css">
div.exp{
padding: 0;
margin: 0;
}
div.loc{
margin-left: 19px;
}
</style>
<script type="text/javascript">
<!--
/**
* Season info *
* 0 Fall: Aug 26 - Nov 1
* 1 Winter Training: Nov 2 - March 1
* 2 Spring: March 2 - May 25
* 3 Summer: May 26 - Aug 25
**/

/**
* Callback function for the GData json-in-script call
* Inserts the supplied list of events into a div of a pre-defined name
*
* @param {json} root is the JSON-formatted content from GData
*/
function processRaces(root) {
displayRaces(root.feed, 'races');
}

//meow
function displayRaces(feed, divId){
var now = new Date();
var season = getSeason(now);
var events = document.getElementById(divId);

//clear "Loading..."
if (events.childNodes.length > 0){
events.removeChild(events.childNodes[0]);
}

//display season
var d = document.createElement('div');
d.appendChild(document.createTextNode(getSeasonText(season)));
events.appendChild(d);

//loop races
for (var i=0; i<feed.entry.length; i++){
var entry = feed.entry[i];

var d = getDate(entry['gd$when'][0].startTime);
if(isCurrSeason(now, d)){
var title = entry.title.$t;
var desc = entry.content.$t;

//set up image and clicking to expand
var div = document.createElement('div');
div.className = 'exp';
var toggle = document.createElement('img');
toggle.src= 'http://timothy.broder.googlepages.com/p.jpg';
toggle.align = 'absmiddle';
toggle.id = i;
toggle.onclick = function() {toggleDiv(this); }
div.appendChild(toggle);
div.appendChild(document.createTextNode(' ' + d.getMonth() + "/" + d.getDay() + ' - '));

// get the href to link to the event
for(var j=0; j<entry['link'].length; j++){
if (entry['link'][j]['type'] == 'text/html' && entry['link'][j]['rel'] == 'alternate'){
var href = entry['link'][j]['href'];
}
}

//we can link to the cal
if (typeof href != 'undefined'){
var link = document.createElement('a');
link.href = href;
link.target = '_blank';
link.appendChild(document.createTextNode(title));
div.appendChild(link);
}
else{ //shouldn't get here but just in case
div.appendChild(document.createTextNode(title));
}

div.appendChild(document.createElement('br'));

events.appendChild(div);

var where = entry['gd$where'][0].valueString;
var tDiv = document.createElement('div'); //div that will be hidden initially
tDiv.id ='id' + i;
tDiv.style['display'] = 'none';
tDiv.className = 'loc';
events.appendChild(tDiv);

if(desc != null && desc != ""){ //we have a desc (should be the teams we're competing against)
var dDiv = document.createElement('span');
dDiv.appendChild(document.createTextNode(desc + " "));
tDiv.appendChild(dDiv);
}
//the location field is populated (hidden at start)
if(where != null && where != ""){ //we have a location
var it = document.createElement('i');
var map = document.createElement('a');
map.target = '_blank';
//link to location on google maps
map.href = 'http://maps.google.com/maps?f=q&hl=en&&q=' + spaceLink(entry['gd$where'][0].valueString);
map.appendChild(document.createTextNode(entry['gd$where'][0].valueString));
it.appendChild(document.createTextNode(" ("));
it.appendChild(map);
it.appendChild(document.createTextNode(")"));
tDiv.appendChild(it);
}
}
}
}

//format the date a little
function getDate(when){
var data = when.split("-");
return new Date(data[0], data[1], data[2]);
}

//return int representation of season
function getSeason(d){
var month = d.getMonth();
var day = d.getDate();
var year = d.getFullYear();

var aug = new Date(year, 8, 25);
var nov = new Date(year, 11, 1);
var mar = new Date(year, 3, 1);
var may = new Date(year, 5, 25);

if(aug < d && d <= nov) return 0; //fall
else if(mar < d && d <= may) return 2; //spring
else if(may < d && d <= aug) return 3; //summer
else return 1; //winter
}

//figure out the current season
function isCurrSeason(now, d){
currSeason = getSeason(now);
season = getSeason(d);

if(currSeason == season){ //same season yes
if(now.getFullYear() == d.getFullYear()){ //same year also, match
return true;
}
if(season == 1 && (d.getFullYear() == now.getFullYear()-1)){ //diff year, prob winter
return true;
}
}
return false;
}

//return text for season
function getSeasonText(season){
if(season == 0) return "Fall Season";
if(season == 1) return "Winter Training";
if(season == 2) return "Spring Season";
if(season == 3) return "Summer Season";

return "no season"; //really shouldn't get here
}

//if the div is hidden show it, if not, hide it
function toggleDiv(where_id){
var div = document.getElementById('id' + where_id.id);
var img = document.getElementById(where_id.id);
if(div != null){
if(div.style.display != 'none'){
div.style.display = 'none';
img.src= 'http://timothy.broder.googlepages.com/p.jpg';

}
else{
div.style.display = 'block';
img.src= 'http://timothy.broder.googlepages.com/m.jpg';
}
}
}

//convert the location so it can be used in a link to google maps
function spaceLink(name){
return name.replace(' ', ',+');
}

//-->
</script>

<div id="races"><p>Loading...</p></div>
<script type="text/javascript" src="http://www.google.com/calendar/feeds/rpicrew@gmail.com/public/full?alt=json-in-script&callback=processRaces&orderby=starttime&singleevents=true&sortorder=ascending&start-min=2007-01-01T00:00:00"></script>

And the XML for the gadget

<?xml version="1.0" encoding="UTF-8"?>
<Module>
<ModulePrefs title="RPI Crew Schedule"
title_url="http://gpowered.net"
author="Tim Broder"
height="150"
width="250"
author_email="timothy.broder@gmail.com"
thumbnail="http://timothy.broder.googlepages.com/RPICrewScheduleThumb.jpg"
description="RPI Crew Race Schedule, links to the RPICrew shared google calendar"
author_photo="http://timothy.broder.googlepages.com/timothybrodersimpsons.png"
author_location="NYC"
author_affiliation="gPowered"
author_link="http://www.gpowered.net"
screenshot="http://timothy.broder.googlepages.com/RPICrewScheduleScreen.jpg"
>
<Require feature="dynamic-height"/>
<Require feature="analytics"/>
</ModulePrefs>
<Content type="html"><![CDATA[


<script>
// Track this gadget using Google Analytics.
_IG_Analytics("UA-793489-6", "/RPICrewScheduleG");
</script>
<style type="text/css">
div.exp{
padding: 0;
margin: 0;
}
div.loc{
margin-left: 19px;
}
</style>
<script type="text/javascript">
<!--
/**
* Season info *
* 0 Fall: Aug 26 - Nov 1
* 1 Winter Training: Nov 2 - March 1
* 2 Spring: March 2 - May 25
* 3 Summer: May 26 - Aug 25
**/

/**
* Callback function for the GData json-in-script call
* Inserts the supplied list of events into a div of a pre-defined name
*
* @param {json} root is the JSON-formatted content from GData
*/
function processRaces(root) {
displayRaces(root.feed, 'races');
}

//meow
function displayRaces(feed, divId){
var now = new Date();
var season = getSeason(now);
var events = document.getElementById(divId);

//clear "Loading..."
if (events.childNodes.length > 0){
events.removeChild(events.childNodes[0]);
}

//display season
var d = document.createElement('div');
d.appendChild(document.createTextNode(getSeasonText(season)));
events.appendChild(d);

//loop races
for (var i=0; i<feed.entry.length; i++){
var entry = feed.entry[i];

var d = getDate(entry['gd$when'][0].startTime);
if(isCurrSeason(now, d)){
var title = entry.title.$t;
var desc = entry.content.$t;

//set up image and clicking to expand
var div = document.createElement('div');
div.className = 'exp';
var toggle = document.createElement('img');
toggle.src= 'http://timothy.broder.googlepages.com/p.jpg';
toggle.align = 'absmiddle';
toggle.id = i;
toggle.onclick = function() {toggleDiv(this); }
div.appendChild(toggle);
div.appendChild(document.createTextNode(' ' + d.getMonth() + "/" + d.getDay() + ' - '));

// get the href to link to the event
for(var j=0; j<entry['link'].length; j++){
if (entry['link'][j]['type'] == 'text/html' && entry['link'][j]['rel'] == 'alternate'){
var href = entry['link'][j]['href'];
}
}

//we can link to the cal
if (typeof href != 'undefined'){
var link = document.createElement('a');
link.href = href;
link.target = '_blank';
link.appendChild(document.createTextNode(title));
div.appendChild(link);
}
else{ //shouldn't get here but just in case
div.appendChild(document.createTextNode(title));
}

div.appendChild(document.createElement('br'));

events.appendChild(div);

var where = entry['gd$where'][0].valueString;
var tDiv = document.createElement('div'); //div that will be hidden initially
tDiv.id ='id' + i;
tDiv.style['display'] = 'none';
tDiv.className = 'loc';
events.appendChild(tDiv);

if(desc != null && desc != ""){ //we have a desc (should be the teams we're competing against)
var dDiv = document.createElement('span');
dDiv.appendChild(document.createTextNode(desc + " "));
tDiv.appendChild(dDiv);
}
//the location field is populated (hidden at start)
if(where != null && where != ""){ //we have a location
var it = document.createElement('i');
var map = document.createElement('a');
map.target = '_blank';
//link to location on google maps
map.href = 'http://maps.google.com/maps?f=q&hl=en&&q=' + spaceLink(entry['gd$where'][0].valueString);
map.appendChild(document.createTextNode(entry['gd$where'][0].valueString));
it.appendChild(document.createTextNode(" ("));
it.appendChild(map);
it.appendChild(document.createTextNode(")"));
tDiv.appendChild(it);
}
}
}
}

//format the date a little
function getDate(when){
var data = when.split("-");
return new Date(data[0], data[1], data[2]);
}

//return int representation of season
function getSeason(d){
var month = d.getMonth();
var day = d.getDate();
var year = d.getFullYear();

var aug = new Date(year, 8, 25);
var nov = new Date(year, 11, 1);
var mar = new Date(year, 3, 1);
var may = new Date(year, 5, 25);

if(aug < d && d <= nov) return 0; //fall
else if(mar < d && d <= may) return 2; //spring
else if(may < d && d <= aug) return 3; //summer
else return 1; //winter
}

//figure out the current season
function isCurrSeason(now, d){
currSeason = getSeason(now);
season = getSeason(d);

if(currSeason == season){ //same season yes
if(now.getFullYear() == d.getFullYear()){ //same year also, match
return true;
}
if(season == 1 && (d.getFullYear() == now.getFullYear()-1)){ //diff year, prob winter
return true;
}
}
return false;
}

//return text for season
function getSeasonText(season){
if(season == 0) return "Fall Season";
if(season == 1) return "Winter Training";
if(season == 2) return "Spring Season";
if(season == 3) return "Summer Season";

return "no season"; //really shouldn't get here
}

//if the div is hidden show it, if not, hide it
function toggleDiv(where_id){
var div = document.getElementById('id' + where_id.id);
var img = document.getElementById(where_id.id);
if(div != null){
if(div.style.display != 'none'){
div.style.display = 'none';
img.src= 'http://timothy.broder.googlepages.com/p.jpg';

}
else{
div.style.display = 'block';
img.src= 'http://timothy.broder.googlepages.com/m.jpg';
}
}
}

//convert the location so it can be used in a link to google maps
function spaceLink(name){
return name.replace(' ', ',+');
}

//-->
</script>

<div id="races"><p>Loading...</p></div>
<script type="text/javascript" src="http://www.google.com/calendar/feeds/rpicrew@gmail.com/public/full?alt=json-in-script&callback=processRaces&orderby=starttime&singleevents=true&sortorder=ascending&start-min=2007-01-01T00:00:00"></script>




]]></Content>
</Module>

HOWTO: Displaying Blogger feeds with Python

This first HOWTO is going to cover how I did the basic structure of the posts section of gPowered: pulling from blogger. The main functions we'll need are to show a range of posts, a single post, and a function to get the total number of posts that are in the blog. The reason I try to get the total number is so the different pages can be bookmarkable. When retrieving a GData feed from blogger. Entry 1 in the feed is going to be the newest post so there is no 'id' that is going to be static, so we'll do a little math to get a number we can later use as the id. I use Django to handle rendering the python to the browser. I probably won't get into other methods.

To start out you'll have to grab the following modules.
- the gdata python client
- ElementTree
- mxDateTime (I used this for date formatting)

We then import the parts the we'll need (you can ignore the Django part for your own methods):

from Django.shortcuts import render_to_response

from gdata import service
import gdata
import atom
import getopt
import sys

from mx import DateTime

One of the first things we're going to have to do is authenticate with google services. There are two ways to do this: AuthSub proxy authentication which has a user login using their own credentials, and ClientLogin username/password authentication where you send a username and password. We will be using ClientLogin. First off, set your login credentials to use later.

#login info
user = 'timothy.broder@gmail.com'
password = '********'

Then we start setting up our call to the service.

#set up service
blogger_service = service.GDataService(user, password)
blogger_service.source = 'gpowered'
blogger_service.service = 'blogger'
blogger_service.server = 'www.blogger.com'
blogger_service.ProgrammaticLogin()

For more info see the blogger developer's guide with python or the Google Account Authentication documentation
After we have authenticated with Google we need to start building up our query to GData, which will return as an ElementTree of data. The first thing you'll need is your blog's id.
You can use the function in the dev guide to help you with this if you don't already know it.

def PrintUserBlogTitles(blogger_service):
  query = service.Query()
  query.feed = '/feeds/default/blogs'
  feed = blogger_service.Get(query.ToUri())

  print feed.title.text
  for entry in feed.entry:
    print "\t" + entry.title.text

After you have the blog id we can start working on the query

blog_id = 413573351281770670
feed_url = '/feeds/%s/posts/default' % str(blog_id)
query = service.Query()
query.feed = feed_url

The below function returns the total number of posts that are in the feed. We can get a small response by sending 0 for the max results. Below is the function and the small response we get from it.

#get the total number of posts for this feed
def get_total(query):
 #query for no posts
 query.max_results = '0'
 query.start_index = '1'
 
 #get back entryless feed
 feed = blogger_service.Get(query.ToUri()) 
 return int(feed.total_results.text)

 1
 0
 1
 Blogger
 Tim
 tag:blogger.com,1999:blog-413573351281770670
 
 
 
 
 gPowered
 2007-07-18T10:55:06.728-05:00

So we get the total number of posts and then we can start pulling data. Lets make a generic function, PostFrom, that can be used to show multiple posts, or just single ones, depending on what you pass to it. The start number that is passed to PostFrom has been set to the first post in the blog is considered to have an id of 1 and the latest post is the same as total_posts. This is useful so if viewers want to bookmark the page they are looking at, the post that is being displayed will not change. The following are the different functions that will make use of it.

#show latest posts
def Posts(request):
 return ListPosts(request, total_posts)

#show posts starting from a certain point 
def ListPosts(request, start):
 start = total_posts - int(start) + 1
 return PostFrom(request, start, show_num)

#show a single post
def Post(request, start):
 start = total_posts - int(start) + 1
 return PostFrom(request, start, 1)
 
def PostFrom(request, start, count):
 #query for count number of posts starting at the given post
 query.max_results = str(count)
 query.start_index = str(start)
 feed = blogger_service.Get(query.ToUri())

now we have all the data we need in the feed variable. Its been turned into an element tree so we don't have to worry about XML parsing here. Every node has become an objects and lists. Objects for single nodes(title), and lists for where there are multiple nodes of the same name (entry, link) At this stage I play with the data a little so it's easier to use in my Django template.

 #normalize data for output
 for entry in feed.entry:
  #get link for template
  entry.my_link = entry.link[0].href
  
  #id for links
  entry.my_id = curr_id
  curr_id -= 1
  
  #format published date
  dt = DateTime.ISO.ParseDateTimeUTC(entry.published.text)
  entry.my_date = dt.strftime('%m/%d/%Y')
  entry.my_time = dt.strftime('%I:%M %p') 

Of course we're going to need next and previous buttons as well. The way we've set up the math with total_posts and the start number, we only have to increment or decrement these by count (the number of posts to display on a page). I also set part of the link, as well as the page title, that I will use in my template.

 prev = total_posts - (start - count) + 1
 if prev > total_posts:
  prev = None
  
  
 next = total_posts - (start + count) + 1
 if next < 1:
  next = None
 
 #showing single post 
 if count == 1:
  link = 'post'
  title = feed.entry[0].title.text
 #listing posts
 else:
  link = 'posts'
  title = 'home'

The final part of the function is a return to the Django framework to populate my template. I'm going to get into the template more in the next post, but you now have all the information you need stored.

 return render_to_response('posts/index.html', {
  'entries': feed.entry,
  'title': title,
  'tag_link': tag_link, 
  'prev': prev,
  'next': next,  
  'link': link,
  'tab_home': True,
  })

To the template!

The first part consists of displaying the post itself, along with its relevant information

{% for entry in entries %}

  <h2><a href="/post/{{ entry.my_id }}">{{ entry.title.text }}</h2></a>
  {{ entry.content.text }}
  <p>Posted by {% for auth in entry.author %}{{ auth.name.text }}{% if not forloop.last %}, {% endif %}{% endfor %} 
  on {{ entry.my_date }} at {{ entry.my_time }}</p>
  {% if entry.category %}<p>Labels: {% for cat in entry.category %}
   <a href="{{ tag_link }}{{ cat.term }}">{{ cat.term }}</a>
   {% if not forloop.last %}, {% endif %}
   {% endfor %}</p>{% endif %}<p><a href="{{ entry.my_link }}">More...</a></p>
   
{% endfor %}

And the second part handles the prev and next links

{% if prev %}
prev
{% else %}
    
{% endif %}
    
{% if next %}
next
{% else %}
    
{% endif %}

{% include 'bottom.html' %}

That's all for now. A working example is the gPowered.net site I'm putting up. All posts on there are getting pulled from this blog. I'm going to get into tags and comments next time, as well as javascript and php versions. Stay tuned!