MongoDB or CouchDB - fit for production?

asked by on

I was wondering if anyone can tell me if MongoDB or CouchDB are ready for a production environment.

I'm now looking at these storage solutions (I'm favouring MongoDB at the moment), however these projects are quite young and so I foresee that I'm going to have to work quite hard to convince my manager that we should adopt this new technology.

What I'd like to know is:

  1. Who is using MongoDB or CouchDB today in a production environment?

  2. How are you using MongoDB/CouchDB?

  3. What problems (if any) did you come across when you adopted this new storage mechanism (and how did you overcome them)?

  4. How did you deal with any migration issues that you had to deal with?

  5. Do you have any good/bad experiences with either of these solutions that you'd like to share?

19 Answers

I don't know anything about MongoDB, but from the CouchDB FAQ:

Is CouchDB Ready for Production?

Yes, see InTheWild for a partial list of projects using CouchDB. Another good overview is CouchDB Case Studies

Also, some links:

We use couchdb in production and have since just before the project went under the Apache umbrella.

We use it to store everything that we might otherwise use a dbms, plus all sorts of unstructured data. Personally, I really like how you can just throw all sorts of data into it and use the views to cull what you don't need depending on the situation.

The hardest part was moving away from the dbms mindset. We wrote our own migration utils when the storage format changed just to be safe, so that wasn't really a problem.

We haven't had any negative experiences yet, but then again we haven't had the setup under any kind of huge load. I think things would work pretty well since we have two slave type servers that replicate from a single master server that gets all of the writes. I'm pretty sure that we don't have to do it that way for replication to work correctly, but it's how we set it up in the beginning and it stuck.

I'm the CTO of 10gen (developers of MongoDB) so I'm a bit biased, but I also manage a few sites that are using MongoDB in production.

businessinsider has been using mongo in production for over a year now. They are using it for everything from users and blog posts, to every image on the site.

shopwiki is using it for a few things including real time analytics and a caching layer. They are doing over 1000 writes per second to a fairly large database.

If you go to the mongodb Production Deployments page you'll see some people who are using mongo in production.

If you have any questions about the scale or scope of production deployments, post on our user list and we'll be more than happy to help.

We use CouchDB to store mobile inbound and outbound messages and to report on this traffic via some custom views that I wrote. The front-end is written in Python. We did not have any real technical issues, and it has been running since the end of December. The only hurdle I encountered was initially thinking in terms of MapReduce, but once I learned how to do that, everything else went smoothly.

I am using CouchDB in production. Currently it stores all those 'optional' fields that weren't in the original DB schema. And right now I am thinking about moving all data to CouchDB.

It's quite a risky step, I admit. Firstly, because it's not v1.0 yet. And secondly, because it is drivespace-hungry. By my calculations, CouchDB file (with indexes) is ~30 times larger than MySQL database with the same rows. But I am pretty sure it will work out just fine.

SourceForge uses MongoDB. See this presentation or read here.

The BBC and meebo.com use CouchDB in production and so does one of my clients. Here is a list of other people using Couch: CouchDB in the wild

The major challenge is to know how to organize your documents and stop thinking in terms of relational data.

CouchDB 0.11 (released at the end of March) is a feature-freeze release for 1.0. This means we'll be maintaining compatibility with the current API for 1.0, so now is a good time to take another look at CouchDB if you haven't in a while.

The CouchDB 0.11 source code release is available here. There are binary installers and other goodies linked here.

We are currently using MongoDB in production as the caching layer as well as storage engine for product importing and manipulating product data. We are an eCommerce company managing over two million products (100+ million attributes), spanning 10+ distributors and without MongoDB, this task would be nearing impossible.

We are running CouchDB as a replacemant for MySQL for our shops (70.0000 items/shop, a total of 4 million attributes of all items, cross connections between items).

Our goals were:

  1. Easy replication from a master-db to several clients with different documents.

  2. Fast pre-calculated data like "how many parts do I have with this attribute and that filter, fitting to those conditions"

facts:

  1. Our shops are now running much faster than with MySQL (and mysql-database needed additionaly 1-3 days of pre-calculating (so updating was twice a month), making the data ready for product counting and filtering, CouchDB needs 5 hours, so we could update product data every night)
  2. Setting up (filtered) data distribution & backups to the shop nodes is fast and easy

but also:

  1. Understanding map/reduce and the limits of not having joins is quite hard
  2. No operation on data like "delete where" or "update where" without external programs
  3. Replication works well, unless there is a problem; then it's really hard to find out what was the reason (for beginners)
  4. The installation of CouchDB without binaries (yes there are a some in the wild, but not for every OS/version) could be hard, if you are not a Linux geek. But the CouchDB Community is helpful (#couchdb), and luckily there are companies out there (cloudant, iriscouch) that offer services from free to big business.
  5. CouchDB is moving forward, so there are a lot of changes (improvements) going on that might change they way you work. But basic things remain stable.

As a result: MySQL as a database for data creation and maintaining is reliable and easy to understand and handle. I think we will not change this. But I also don't want to miss the power of CouchDB views and the ease of replication setup.

Production couches sometimes caused trouble after months of work due to misconfiguration and forgotten logrotates (view building takes too long or hangs, replication stops), but never lost data, and always could be easily reset.

We are using MongoDB in production in our mobile backend service namely Netmera. We are using it to store all user and content data.

We are currently using mongodb as an file storage service for our collaboration over LAN. Also, projects like trello are using mongodb as their backend datastore. I have used couchdb earlier, but not in production capacity.

I have been using CouchDB in production for almost 2 years now. There's no migration work as the project started of directly with CouchDB implementation. It serves as a database that stores the data of a single electronic product from beginning till packaging.

Since we are selling sensor with a demand on high accuracy, we do a lot of test at different stage and all these will be stored into one document on CouchDB.

There's some learning curve that I learnt from my experience, which is to make full use of the views (or also known as permanent views). Views should be "small filter" of a fraction of the Database that will be called often.

My CouchDB databse is not as crazy as other gigantic company. But so far, I'm still doing fine. Currently I'm having 24000 documents at 700MB.

Feature from CouchDB that I like is 'replication', 'store revisions of a document'.

I'd read a lot of good reviews on MongoDB and I will want to try it if there's a chance.

We are using mongodb in production for

www.beachfront.io - close to 5k write request per sec www.beachfrontbuilder.com - 500 read/write request per sec, maintain 10m users data & olap.

The only challenge faced around archiving of data, we overcome by implementing our custom component.

This question has already accepted answer but now a days one more NoSQL DB is in trend for many of its great features. It is Couchbase; which runs as CouchbaseLite on mobile platform and Couchbase Server on your server side.

Here is some of main features of Couchbase Lite.

Couchbase Lite is a lightweight, document-oriented (NoSQL), syncable database engine suitable for embedding into mobile apps.

Lightweight means:

Embedded—the database engine is a library linked into the app, not a separate server process. Small code size—important for mobile apps, which are often downloaded over cell networks. Quick startup time—important because mobile devices have relatively slow CPUs. Low memory usage—typical mobile data sets are relatively small, but some documents might have large multimedia attachments. Good performance—exact figures depend on your data and application, of course.

Document-oriented means:

Stores records in flexible JSON format instead of requiring predefined schemas or normalization. Documents can have arbitrary-sized binary attachments, such as multimedia content. Application data format can evolve over time without any need for explicit migrations. MapReduce indexing provides fast lookups without needing to use special query languages.

Syncable means:

Any two copies of a database can be brought into sync via an efficient, reliable, proven replication algorithm. Sync can be on-demand or continuous (with a latency of a few seconds). Devices can sync with a subset of a large database on a remote server. The sync engine supports intermittent and unreliable network connections. Conflicts can be detected and resolved, with app logic in full control of merging. Revision trees allow for complex replication topologies, including server-to-server (for multiple data centers) and peer-to-peer, without data loss or false conflicts. Couchbase Lite provides native APIs for seamless iOS (Objective-C) and Android (Java) development. In addition, it includes the Couchbase Lite Plug-in for PhoneGap, which enables you to build iOS and Android apps that you develop by using familiar web-application programming techniques and the PhoneGap mobile development framework.

You can explore more on Couchbase Lite

and Couchbase Server

This is going to the next big thing.

Speaking production, seamless failover/recovery both require a baby sitter
1- Couchbase, there is no seamless failover/recovery, manual intervention is required.
rebalancing takes too much time, too much risk if more than one node get lost.

2- Mongo with shards, data recovery from loosing a config server, is not an easy task

Adobe is using MongoDB for their upcoming release of Adobe Experience Manager (formerly Day CQ) as the core DB engine.

Several client's at the agency I work at are using CouchDB on projects for large clients.

Both are great and viable DBs, in my opinion. :)