|
I'm getting better at querying mongo and I hate it.
|
![]() |
|
![]()
|
# ? Feb 17, 2025 02:28 |
|
CRIP EATIN BREAD posted:I've never met a person asking about a K/V store that had a workload that even came close to requiring something other than a postgres table with the default database settings. poo poo. This is a fantastic time for this discussion to pop up, since I'm supposed to start implementing something like this soon. I managed to convince my team to go serverless for a new feature that's small but could have high volume. We were already planning to use a an AWS Lambda thing to handle intake, but I suggested using DynamoDB as a K/V store rather than doing some caching strategy to lookup data from our actual server+db. The K/V store will likely only be storing a few thousand rows at most for the foreseeable future. I know I'm a terrible programmer, but am I the terrible programmer?
|
![]() |
|
pseudorandom posted:poo poo. Seems fine. I’d had to know more about your use case and proposed design to determine how terrible of a programmer you are.
|
![]() |
|
redleader posted:what rdbms and full-text search thing do you prefer/have used for this? We support Oracle, SQL Server and Azure SQL with Solr for text searching using JMS to update the Solr cluster on CRUD. The RDBMS is not fully within our control which is why we support multiple vendors there..The JMS we were already using for something else so it all slotted together pretty nicely. We have one outstanding issue of indexing multilingual data correctly but we have a solution for that we think.
|
![]() |
|
I've never used Solr, but I used elasticsearch back in its pre-1.0 days, and hoo boy was it not production ready. I will say that I've been using it again lately and it's a lot more stable than it used to be, and has suited needs so far. The thing I still hate about it is the json-based DSL for querying. It's fairly obtuse and I almost always have to go to documentation even to do something very simple. No idea how that compares to Solr, and I'd like to spend time checking that out, next chance I get.
|
![]() |
|
I just got a robocall from Belize. which of you is doing this?
|
![]() |
|
![]()
|
![]() |
|
Finster Dexter posted:I've never used Solr, but I used elasticsearch back in its pre-1.0 days, and hoo boy was it not production ready. I will say that I've been using it again lately and it's a lot more stable than it used to be, and has suited needs so far. The thing I still hate about it is the json-based DSL for querying. It's fairly obtuse and I almost always have to go to documentation even to do something very simple. elasticsearch got a lot more stable, yeah, especially with the new distributed consensus layer introduced in 7.0 and the newish sequence number stuff yeah the json dsl is kind of obtuse sometimes but if you’re okay using the free default distribution (vs the OSS version) it speaks sql now (tho still no joins)
|
![]() |
|
leper khan posted:I just got a robocall from Belize. which of you is doing this? couldnt be me; i "died", am "dead"
|
![]() |
|
working on my dumb librarycode:
if your compiler doesnt support anonymous unions go to hell. e: coupla fixes ee: api design when drunk is hard matti fucked around with this message at 23:54 on Jul 26, 2019 |
![]() |
|
matti posted:if your compiler doesnt support anonymous unions go to hell.
|
![]() |
|
The freopen function first attempts to close any file that is associated with the specified stream. Failure to close the file is ignored. The error and end-of-file indicators for the stream are cleared. pff. like you would hope a failure would set the error flag but no actually the exact opposite happens. what a world
|
![]() |
|
im sure it all works in practice but when writing code examples for documentation id rather not do anything platform specific since you soon get into a chicken and egg situation
|
![]() |
|
Arcsech posted:elasticsearch got a lot more stable, yeah, especially with the new distributed consensus layer introduced in 7.0 and the newish sequence number stuff also you can do a major version upgrade without a cluster restart. this is new in...6 I think. maybe 7. 1.x was pretty drat painful, 2.x not so much, 5.x pretty ok except for the restart issue
|
![]() |
|
here's something i think would work good but haven't tried. set up a replication client for mysql/postgresql that turns binlog into kafka records. feed that into solr/elasticsearch. i'm sure one can think of lots of uses for a db changelog besides updating search yeah i'm far from the first one to think of that. debezium does exactly what i was thinking of
|
![]() |
|
lol uhh its a pretty common use case, it even has an acronym bro (change data capture)
|
![]() |
|
also dont use kafka thx
|
![]() |
|
only other time i've ever encountered that acronym it was plastered on an oracle product so i assumed it was dumb bullshit
|
![]() |
|
there’s even an app for it called kafka connect
|
![]() |
|
Nomnom Cookie posted:only other time i've ever encountered that acronym it was plastered on an oracle product so i assumed it was dumb bullshit it's worth looking at to see if your db architecture is hosed up but as a general rule regarding oracle your instinct is correct
|
![]() |
|
matti posted:The freopen function first attempts to close any file that is associated with the specified stream. Failure to close the file is ignored. The error and end-of-file indicators for the stream are cleared. reporting failures when closing a file is pretty pointless
|
![]() |
|
Plorkyeran posted:reporting failures when closing a file is pretty pointless what
|
![]() |
|
what sort of errors are you expecting to get from fclose() that you will be able to actually do anything with? if you want to handle errors from the implicit flush just call flush explicitly first.
|
![]() |
|
are you saying the fclose could succeed even though the flush didn't not trying to argue here, i sincerely think i'm confused e: if youre saying the most significant error message will come from the flush, i guess we're on the same page DaTroof fucked around with this message at 03:51 on Jul 27, 2019 |
![]() |
|
always make sure your fids are clunked, dammit
|
![]() |
|
DaTroof posted:are you saying the fclose could succeed even though the flush didn't freopen() does the following three steps: 1. flush the file 2. close the file 3. open a new file the actual close step can only fail due to being given an invalid fd, so there aren't any actual errors it can report. flushing the file can fail even when given valid input, but the problem with reporting errors from that step in freopen() is that there's overlap between the error codes of write() and open(); if you get an EIO from freopen() you wouldn't know if it failed to flush, or if it failed to create the file you asked it to open. if you want to handle errors when flushing you need to instead call fflush() first, and after that succeeds it's no longer possible for fclose() to fail.
|
![]() |
|
close() can fail with EIO
|
![]() |
|
[EIO] A previously-uncommitted write(2) encountered an input/output error. you don't have a previously-uncommited write if you explicitly commit your writes before reopening
|
![]() |
|
EIEIO
|
![]() |
|
there's a big stink rn about linux's fsync(2) error reporting behavior being dumb and backwards and causing data corruption in postgres long story short if a block device IO error causes a write to be lost then the next fsync fails but after that the error is considered "reported" and all subsequent fsyncs succeed.
|
![]() |
|
Sapozhnik posted:there's a big stink rn but enough about my posting
|
![]() |
|
Sapozhnik posted:there's a big stink rn about linux's fsync(2) error reporting behavior being dumb and backwards and causing data corruption in postgres the really awful part is that unless you're running a really recent kernel version the "next" fsync() may be one in a different process, so you can call write() immediately followed by fsync(), get success from both of them, and still have the write be lost because someone else ate your error
|
![]() |
|
lol Linux is so bad
|
![]() |
|
Bloody posted:lol computers is so bad
|
![]() |
|
can't wait for the inevitable Linus email where he repeatedly insults the postgres devs' intelligence for daring to find a problem with the kernel
|
![]() |
|
Bloody posted:lol Linux is so bad
|
![]() |
|
redleader posted:can't wait for the inevitable Linus email where he repeatedly insults the postgres devs' intelligence for daring to find a problem with the kernel linux tech tips: the database is bad!
|
![]() |
|
Bloody posted:lol Linux is so bad
|
![]() |
|
Plorkyeran posted:the really awful part is that unless you're running a really recent kernel version the "next" fsync() may be one in a different process, so you can call write() immediately followed by fsync(), get success from both of them, and still have the write be lost because someone else ate your error ah gently caress is there a nice writeup somewhere on this?
|
![]() |
|
![]()
|
# ? Feb 17, 2025 02:28 |
|
fsucc()
|
![]() |