nosql mongodb replicaset autosharding, data loss while running an experiment

I was continuously inserting data into autosharded mongodb. (not updating, only insertion )

Experimental setup 2 shards, each shard has 2 nodes.

in shard1 -> node1 (primary), node2 in shard2 -> node3 (primary), node4

in one shard I brought secondary node (node2) down -> still the data got inserted properly Now I brought the primary(node1) also down -> complete shard is down now

Surprisingly the whole mongo setup was not taking any writes.

there is another twist in the story. Now I have brought the node2 up, as this is the only node available, it beccame primary and started accepting data.

After some 10 mins I have brought node1 also up.

But when I checked the data that I have inserted into node1 (when node2 is down) was lost :(.

    s1             s2
   (n1,n2)       (n3,n4)  
   ---------     ---------
1. (up,up)       (up, up)  -> data was getting inserted
2. (up,down)     (up, up)  -> data was getting inserted properly (but got lost :(  )
3. (down, down)  (up, up)  -> no data inserts were happening
4. (down, up)    (up, up)  -> started taking data again
5. (up, up)      (up, up)  -> n1 picked data from n2, but couldn't give the data to n2

in the last step, when I brought n1 up, it got the new 开发者_运维问答records from n2, but the that n1 got during step 2 above were completely lost :(.

Is this expected behaviour of mongo?

Someone please help me out, I can run more experiments if required :).

Thanks in advance, Vivekananda Tadala

If you bring down one member in a two member replica set, the other will stop accepting writes. See this explanation of why you need three members and always use safe mode when you're playing around.

Replication in MongoDB is one-directional: a slave copies data from a more up-to-date slave or the master. You can't have two nodes copying data from each other (yet). You may be interested in ensuring write replication in the meantime.

Inserting into chunks on a downed shard won't work, but you can insert into chunks on shards that are on the other shard. You can use db.printShardingStatus() to see which chunks are where.