Sunday, February 11, 2007

The Google grid

Ever wondered if Google requires all that space that it promises the Gmail users? I mean does Google need 2 GB * (Number of Gmail users) amount of space on the disk? Well this discussion will prove that he does not need all that space and can still promise his users the 2 GB quota. But this is also an attempt to figure out how much roughly could be the minimum storage capacity required by Google for its Gmail users.

Assume that 'L' is the storage limit imposed by Google for its mail users.
Let S = Rate at which Google's storage increases per month
Let Ri = Initial average rate at which email volume increases in a mailbox
Let r = average rate at which email volume increases in a mailbox at any given time.

As the limit to mail per person is limited by L, it can be assumed that the average mail volume increases exponentially. It starts with an initial rate Ri and the mailbox finally reaches the volume limit.
Thus Consumption C per person = C = L(1-exp(-Ri*t/L)) --- (1)

The rate of mailbox size increase = r is the differentiation of (1) wrt t
r= Ri*exp(-Ri*t/L)

At any given point, the rate of Google's storage space increase should be greater than or equal to r.
Assuming equality, S=r,
S=Ri*exp(-Ri*t/L)

The current value of L is 2816 MB
Ri can be assumed to be 40 MB/Month/Person
At t = 24 months, S turns out to be 28.4 MB/Month/Person

Assume that there are roughly 40000000 Gmail users.
The the average storage capacity added to Google every month should be
40000000*28.4 = 1.1 Peta Bytes per month! (Peta = 10^15)

For the past 2 years, Google should have then amassed 24*1.1 = 26.4 Peta Bytes of storage space! As Google's storage is triple redundant, it should have around 80 Peta bytes of Hard disk space!!

Well there are too many approximations and assumptions in here. But it gives the magnitude of space that you should have/add to promise a free storage space of 2.8 GB.

This shows that if all Gmail users would use up all Google's space today, each user would end up getting only about 660 MB of space!