Python script to fetch messages from facebook account

10/14/2012 Posted by garran


We are going to write a python script which on execution will fetch unread messages from your facebook account. My motivation behind writing this script is my pathetic slow internet connection due to which sometimes it took me around 1-3 minutes to just check if there are any new messages. So, I thought why not write a python script to achieve this task. The python facebook API we are going to use is facepy. I would try to be as descriptive as possible while writing this blog post.

First, of all download the zip file from the facepy github page or those who are familiar with git can use git clone to clone the repository. Now, open your terminal/DOS prompt and move to the directory where you have either extracted the zip file contents or cloned the repository and type
    python setup.py install (Windows Users)

    sudo python setup.py install (Linux Users)

Make sure you installed the latest version from github because the pip one is not updated.

Now, there are two possible ways to query facebook for messages. We will discuss both ways and see the shortcomings of one over the other.

Method 1

Facebook uses access token to authenticate a user and gives it the ability to know the identity of a facebook user. There are mainly 3 types of access token user, page, app. The one we are interested here is the user access token. Now, to get the user access token visit https://developers.facebook.com/tools/explorer/  and click on Get access token and it will ask for some permission so go to extended permission tab and grant it read_mailbox permission. Read more about extended permissions in facebook docs.

After the above step you will be granted a new access token and also graph explorer will show you the output of  /yourid?fields=id,name which will be your facebook id and your name. If you are not able to see the above output make sure you followed the steps correctly.

So, now you have got the access token which we will use to authenticate ourself against facebook.  Remember I talked about discussing the shortcomings of one of the method so lets discuss it. Again visit https://developers.facebook.com/tools/explorer/  and you will see facebook will automatically fill the access token field. Now, click on the Debug button you see next to token field and facebook will show some information related to your access token. The information we are interested in is Expires field which will show you something like

           Expires xxxxx (in about an hour)

So, by now you might have guessed the shortcoming of method 1 i.e. our access token will expire in about an hour or two. What this means is that every two hours we have to fetch a new token if we want to continue using our python script otherwise it will raise an OAuthError. The token we got above is called short lived access token valid for approx. 2 hours at most.

Method 2

Facebook provide an option to get access tokens which have a long-live expiration time. Currently the long-lived user access token will be valid for 60 days while the short-lived user access tokens are valid from 1 to 2 hours.  Now to extend a short-lived token we are going to use get_extended_access_token method of facepy. If you observe the above method it requires an application_id and application_secret_key along with the token.

So, let's get them:

Now, I am not going to go into the details of how to create a new app on facebook because its fairly easy and you can follow any guide you want but keep in mind to give proper permissions to the app.

Once you have created a new app you will be able to see App ID/API Key and App secret mentioned at the application's facebook page. Once again visit https://developers.facebook.com/tools/explorer/  and this time select application as your app_name and copy the access_token. Now, substitute the value of short-lived access token, app_id, app_secret in the below code


Here app_name is MessageFetcher


#Imports from facepy api
from facepy.utils import get_extended_access_token

#App key/APP ID and App_secret from https://developers.facebook.com/apps
app_id = 'xxxxxxxxxxxxxx'
app_secret = 'yyyyyyyyyyyy'

#Token got by selecting app_name from the drop-down list.
short_lived_access_token = "token"

long_lived_access_token, expires_at = get_extended_access_token(
    short_lived_access_token,
    app_id, app_secret)

print long_lived_access_token
print expires_at
Save it as extended_token.py and run it as python extended_token.py and you will see it will output an extended token along with it's expiration date. My output is:
      2012-12-12 20:39:14.679582 # Approx. two months
You can also check whether the token you got is long-lived or not by using the graph explorer debug method. So, we have got a token which has an expiration period of 60 days.

Let's get back to our main objective to fetch messages from facebook. We are going to read messages by querying the thread table using FQL.

From facebook docs:

FQL:

         FQL stands for facebook query language. FQL, enables you to use a SQL-style interface to query the data exposed by the Graph API.

Thread Table:

         Information about message threads in a user's inbox as represented in FQL. To access the thread table you need an access token with extended read_mailbox permissions.

FQL we are going to use is
SELECT snippet, snippet_author, recipients FROM thread WHERE folder_id = 0 and unread != 0
You can get the meaning of columns mentioned in the fql query by visiting the thread page. So, our work is done just use the below code substitute the required value and run it.

#!/usr/bin/python
# -*- coding: utf-8 -*-

import sys

from facepy import GraphAPI
from facepy import exceptions

#Acces token with expire time = 60days
LONG_LIVE_ACCESS_TOKEN = 'token'

#Facebook app id and secret from http://developer.facebook.com/apps
APP_ID = 'xxxxxxxxxxxx'
SECRET_KEY = 'yyyyyyyyyyy'

def user_id_to_username(userid):

    """ Function to convert facebook USERID to username. """

    if userid is not None:

        userid = '/{0}'.format(userid)
        try:
            return  graph.get(userid)['name']

        except (exceptions.FacebookError, exceptions.OAuthError) as e:
            print e.message
            sys.exit(0)

def get_message_author(message_list):

    """ Function to get the author of message."""

    return user_id_to_username(message_list['snippet_author'])


def get_message_author_id(message_list):

    """ Function to get the author ID."""

    return message_list['snippet_author']


def get_message_body(message_list):

    """ Function to get the message or message body."""

    return message_list['snippet']


def get_recipients_list(message_list):

    """ Function to get all the recipients of a message."""

    author = get_message_author_id(message_list)
    temp = message_list['recipients']

    #Facebook includes author as a recipient too. So, we are going to remove it from the
    #list of recepients.
    temp.remove(author)
    return ", ".join(map(user_id_to_username, temp))


def pretty_print(message_list):

    """ Function which prints the o/p in a specified format."""

    for message in message_list:

        print "from:     ", get_message_author(message)
        print "to:       ", get_recipients_list(message)
        print "Message:  ", get_message_body(message)
        print "-" * 140


#Creating facebook GraphAPI object using the long live token.
graph = GraphAPI(LONG_LIVE_ACCESS_TOKEN)

#Output of the facebook query language(FQL)
#This FQL queries for message body, author, recipients for unread messages.
try:
    json_output = graph.fql(
        'SELECT snippet, snippet_author, recipients FROM thread WHERE folder_id = 0  and unread != 0 Limit 4')
except exceptions.OAuthError as e:
    print e.message
    sys.exit(0)

#Extracting the data content part from the returned json response. The data part will
#contain snippet, author, recepients.
message_list = json_output['data']

#Checking if empty or not.
if message_list:
    pretty_print(message_list)
else:
    print "No New Messages"
    sys.exit(0)
The output will look like:
garran@garran:~$ python facebook_message.py 
from:      TestUser1
to:        TestUser2
Message:   This is a test message.
---------------------------------------------------------------------------------
from:      TestUser2
to:        TestUser1
Message:   Just Checking.
---------------------------------------------------------------------------------
Source code is available at bitbucket.

16 comments:

  1. Hello, great post!
    I'm new to Python and Facebook API.. so i was wondering if you could help me: istead of reading just the first line of the message.. how can i pull the whole conversation?
    Thanks

    ReplyDelete
    Replies
    1. AFAIK, currently this is not possible because if you take a look at facebook's documentation of thread it mentions that one can query about snippet which is " A short bit of text from the most recent message. "

      Delete
    2. ok thanks =D

      Delete
  2. This is so cool!! Wish I know a lot more python

    ReplyDelete
  3. (#298) Requires extended permission: read_mailbox

    I dont understand the error. How to get an access token with extended read_mailbox permissions? Thank you very much. If possible i would like to have a detail guide. please, ! I beg you

    ReplyDelete
    Replies
    1. You probably have to change you app permissions, so that you can read the mailbox. Once you did that, allow the app on your account, should do.

      Delete
  4. Hi I get this running the messenger script:

    (#12) fql is deprecated for versions v2.1 and higher

    ReplyDelete
    Replies
    1. Hi,

      Fql is deperected in newer facebook sdv v2.1. You need to use Graph API to achieve the same.

      Please look at facepy documentation to achieve the same.

      Thanks,
      Garran

      Delete
    2. Will you be kind to drop few (one) line for that error
      Fql is deperected in newer facebook sdv v2.1

      Delete
  5. how to extract user hidden data in all social website using script ?

    ReplyDelete
  6. It'd be really nice if you could update your content so that it can be used for Graph API Explorer v2.8 (and Python 3, if possible). Thanks :D

    ReplyDelete
  7. As of August 8, 2016, FQL will no longer be available and cannot be queried. To migrate your app, use the API Upgrade Tool to see the Graph API calls you can make instead.

    Any idea what must be done to overcome this

    ReplyDelete
  8. burilia-ji_Bridgeport Keith Carouthers
    Link
    essoifracal

    ReplyDelete