Sleepy.Mongoose: A MongoDB HTTP Interface

The first half of the MongoDB book is due this week, so I wrote a REST interface for Mongo (I’m a prolific procrastinator).  Anyway, it’s called Sleepy.Mongoose and it’s available at https://github.com/10gen-labs/sleepy.mongoose.

Installing Sleepy.Mongoose

  1. Install MongoDB.
  2. Install the Python driver:
    $ easy_install pymongo
  3. Download Sleepy.Mongoose.
  4. From the mongoose directory, run:
    $ python httpd.py

You’ll see something that looks like:

=================================
|      MongoDB REST Server      |
=================================

listening for connections on http://localhost:27080

Using Sleepy.Mongoose

First, we’re just going to ping Sleepy.Mongoose to make sure it’s awake. You can use curl:

$ curl 'http://localhost:27080/_hello'

and it’ll send back a Star Wars quote.

To really use the interface, we need to connect to a database server. To do this, we post our database server address to the URI “/_connect” (all actions start with an underscore):

$ curl --data server=localhost:27017 'http://localhost:27080/_connect'

This connects to the database running at localhost:27017.

Now let’s insert something into a collection.

$ curl --data 'docs=[{"x":1}]' 'http://localhost:27080/foo/bar/_insert'

This will insert the document {“x” : 1} into the foo database’s bar collection. If we open up the JavaScript shell (mongo), we can see the document we just added:

> use foo
> db.bar.find()
{ "_id" : ObjectId("4b7edc9a1d41c8137e000000"), "x" : 1 }

But why bother opening the shell when we can query with curl?

$ curl -X GET 'http://localhost:27080/foo/bar/_find'
{"ok": 1, "results": [{"x": 1, "_id": {"$oid": "4b7edc9a1d41c8137e000000"}}], "id": 0}

Note that queries are GET requests, whereas the other requests up to this point have been posts (well, the _hello can be either).

A query returns three fields:

  • “ok”, which will be 1 if the query succeeded, 0 otherwise
  • “results” which is an array of documents from the db
  • “id” which is an identifier for that particular query

In this case “id” is irrelevant as we only have one document in the collection but if we had a bunch, we could use the id to get more results (_find only returns the first 15 matching documents by default, although it’s configurable). This will probably be clearer with an example, so let’s add some more documents to see how this works:

$ curl --data 'docs=[{"x":2},{"x":3}]' 'http://localhost:27080/foo/bar/_insert'
{"ok" : 1}

Now we have three documents. Let’s do a query and ask for it to return one result at a time:

$ curl -X GET 'http://localhost:27080/foo/bar/_find?batch_size=1'
{"ok": 1, "results": [{"x": 1, "_id": {"$oid": "4b7edc9a1d41c8137e000000"}}], "id": 1}

The only difference between this query and the one above is the “?batch_size=1″ which means “send one document back.” Notice that the cursor id is 1 now, too (not 0). To get the next result, we can do:

$ curl -X GET 'http://localhost:27080/foo/bar/_more?id=1&batch_size=1'
{"ok": 1, "results": [{"x": 2, "_id": {"$oid": "4b7ee0731d41c8137e000001"}}], "id": 1}
$ curl -X GET 'http://localhost:27080/foo/bar/_more?id=1&batch_size=1'
{"ok": 1, "results": [{"x": 3, "_id": {"$oid": "4b7ee0731d41c8137e000002"}}], "id": 1}

Now let’s remove a document:

$ curl --data 'criteria={"x":2}' 'http://localhost:27080/foo/bar/_remove'
{"ok" : 1}

Now if we do a _find, it only returns two documents:

$ curl -X GET 'http://localhost:27080/foo/bar/_find'
{"ok": 1, "results": [{"x": 1, "_id": {"$oid": "4b7edc9a1d41c8137e000000"}}, {"x": 3, "_id": {"$oid": "4b7ee0731d41c8137e000002"}}], "id": 2}

And finally, updates:

$ curl --data 'criteria={"x":1}&newobj={"$inc":{"x":1}}' 'http://localhost:27080/foo/bar/_update'

Let’s do a _find to see the updated object, this time using criteria: {“x”:2}. To put this in a URL, we need to escape the ‘{‘ and ‘}’ characters. You can do this by copy-pasting it into any javascript interpreter (Rhino, Spidermonkey, mongo, Firebug, Chome’s dev tools) as follows:

> escape('{"x":2}')
%7B%22x%22%3A2%7D

And now we can use that in our URL:

$ curl -X GET 'http://localhost:27080/foo/bar/_find?criteria=%7B%22x%22%3A2%7D'
{"ok": 1, "results": [{"x": 2, "_id": {"$oid": "4b7edc9a1d41c8137e000000"}}], "id": 0}

If you’re looking to go beyond the basic CRUD, there’s more documentation in the wiki.

This code is super-alpha. Comments, questions, suggestions, patches, and forks are all welcome.

Note: Sleepy.Mongoose is an offshoot of something I’m actually supposed to be working on: a JavaScript API we’re going to use to make an awesome sharding tool.  Administrating your cluster will be a point-and-click interface.  You’ll be able to see how everything is doing, drag n’ drop chunks, visually split collections… it’s going to be so cool.

  • kristina1

    Yeah, probably. It’s not really a REST interface, it’s just an HTTP interface.

  • kajal

    can I insert a json file into mongoDb collection using this REST API?

  • kristina1

    Yes, as either blobs or structured data.

  • antonimmo

    I solved the CORS problem by editing “http.py”, replacing

    response_headers = []
    with
    response_headers = [[‘Access-Control-Allow-Origin’,’*’]]

  • Antonio Quintana

    Hey, hi!

    Do you know how to enable gzip compression if requested by the client?

  • kristina1

    It looks like there is no trivial way (http://stackoverflow.com/questions/9622998/how-to-use-content-encoding-gzip-with-python-simplehttpserver), but if you’d like to implement one of the options listed in that thread, I’d be happy to merge in the patch.

  • sowjanya

    Hi kristina,
    I am unable to run 4th step python httpd.py command it shownig error like this ImportError: No module named bson.son

  • kristina1

    Have you installed pymongo?

  • Wojciech Bogucki

    Great Toturial, I was playing around with few MongoDB HTTP/REST Interfacases and this is the one which I was abe to configure quite fast

  • Wojciech Bogucki

    Guys I cannot insert data into the database, its showing the error couldnt parse json

  • kristina1

    Python is very picky about JSON formatting. Make sure http://jsonlint.com/ thinks it’s valid.

  • Amir Mursal

    How to combine two collections in Sleepy Mongoose?

  • kristina1

    You cannot do a join using MongoDB, regardless of driver. See http://stackoverflow.com/questions/2350495/how-do-i-perform-the-sql-join-equivalent-in-mongodb

  • Amir Mursal

    How we will achieve MAP REDUCE in sleepy.mongoose?

  • Mathumitha

    Hi,

    I get couldnt parse json error when inserting data. I tested the json with the link you had given and it says valid. I am using the below curl in windows command prompt
    curl –data docs={“x”:”1″} http://localhost:27080/test/firstCollection/_insert

  • Mathumitha

    Hi Kristina,

    I face 2 issues with sleepy mongoose.

    1. I have downloaded sleepy mongoose and i am able to do a get request but when i try to insert data i face issue
    ‘couldn’t parse json’. I use the below curl command
    curl –data docs=[{“x”:1}] http://localhost:27080/test/first/_insert

    I also tried using the rest client in mozila firefox but did not get any response. In the httpd.py logs i found a not indexable error.

    2. I have enabled authentication for mongodb. How can the credentials be passed for a find or insert.

    Can you please suggest a solution

    Thanks!!

  • Mathumitha

    Hi Kristina,

    I see a curl command for authenticate. Is there any way in sleepy mongoose to close an existing connection after querying

    Thank you

  • kristina1

    You can call http://docs.mongodb.org/manual/reference/command/logout/ to de-authenticate. I don’t think there’s an explicit disconnect command.

  • Mathumitha

    Thanks Kristina.
    Is there any url or http call to logout similar to _authenticate?

  • kristina1
  • Mathumitha

    For any post using curl and windows, i get json cannot be parsed error. I tried
    curl –data “cmd={logout : 1}” http://localhost:27080/test/_cmd
    curl –data ‘docs=[{“x”:1}]’ “http://localhost:27080/test/firstCollection/_insert”

  • Mathumitha

    Thanks Kristina. With your suggestion i am able to achieve a solution.

kristina chodorow's blog