Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
beuges
Jul 4, 2005
fluffy bunny butterfly broomstick
I hope I'm not reviving this thread for nothing...

I'm building something where I have some devices at different sites running node, which need to keep in sync with each other at the same site as well as a cloud backend. The solution we've come up with is to use PouchDB on each device, syncing with each other as well as a CouchDB instance in the cloud. We've got the devices syncing with each other, and using a selector filter on the CouchDB sync, the devices only replicate documents with a matching site id. I will then have a service on the backend keeping a continuous _changes feed open on the CouchDB, so that records from the devices can get processed into the actual backend.
The reason for the devices syncing via a selector filter instead of each site just having its own db is because we expect to have tens of thousands of sites, and I'm expecting that keeping open that many connections open to a _changes feed on each db is going to be not great.

So:
- am I right in choosing to have a single db instead of a db per site, given that I need to be watching for changes in real time?
- is there a better way to do this instead of using the _changes feed? can I configure some sort of callback function in a design document that can post the document data onto a queue or something, so I don't need to watch _changes?
- if the single db is the way to go, then is it possible to configure CouchDB to block attempts to replicate without a selector filter specified? I'm assigning unique credentials to each device, but I want to prevent a rogue device from syncing the entire DB and seeing data from other sites that it should not be able to see

Ideally I'd rather have a unique database per site so credentials prevent a rogue device from being able to pull down data from another site's db, and have Couch drop the incoming documents onto a queue that my service can watch instead of the _changes feed, but a) I'm not sure if that's possible or how to do it, and b) I'm not sure if that would be better than just monitoring a single _changes feed, and c) I'm not sure if monitoring a single _changes feed instead of tens of thousands of _changes feeds will have a performance impact in the first place.

Please guide me, dear goons

Adbot
ADBOT LOVES YOU

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply