PythonDB

I've written earlier about my frustrations with mysql and memcache. Instead of just complaining, I'll now try to offer some ideas on what might be a better alternative for web applications.

As a first degree of approximation, consider a server that accepted short snippets of python from clients and executed them. You could potentially have a very large amount of RAM and an even bigger swap file to provide hundreds of gigabytes of storage via the heap. Instead of writing "select * from users where id = 'bob'" you might instead say "users['bob']" and get back a json encoded version of whatever object was stored in the dict 'users' under key 'bob'. The snippets might also be more complicated like defining classes, using loops, etc.

You would no longer have to think of the world as only tables (mysql) or key/value pairs (memcache). Instead you can use the same data representations you're already using in your programs. For example, if you wanted a message queue you could simply create a class that has a list, push/pop methods, and a lock for synchronizing it. It's hard to get much simpler than that!

Speaking of locks, you would control the synchronization instead of having the database implicitly try to do it with transactions. As I've written before, I'm very skeptical about using transactions to build modular software.

Replication could work by having the same python snippets sent to all copies of the database. So long as the snippets were deterministic and they were applied in the same order, all of the replicas would end up with the same data.

Obviously, there are some (big!) disadvantages to what I've proposed. There's no persistence, though a python interpreter could manage a persistent paged heap. There would have to be different options for how deeply to serialize results and when to return proxies instead of the actual object values. Different levels of access control may have to be added in some way.

You probably wouldn't have CPython and its full library embedded in some sort of server but instead have a separate implementation of the python language with its own memory manager, concurrency, etc.