|
Symbolic Butt posted:the weirdest thing about han unification is how they didn't even bother to put a language variant tag/modifier (like you can specify skintones with a modifier on emojis nowadays?)
|
![]() |
|
![]()
|
# ? Feb 17, 2025 21:11 |
|
pseudorandom name posted:they introduced the language tags in Unicode 3.1, deprecated them in 5.1, and removed them entirely in 8 mystes posted:As I said earlier, they do have variation selectors for the different forms of the characters now. ok so it's still a confusing mess? yeah, seems like han unification was a great technical decision you guys...
|
![]() |
|
this discussion is getting out of han
|
![]() |
|
ty for the additional context and details
|
![]() |
|
Symbolic Butt posted:ok so it's still a confusing mess? yeah, seems like han unification was a great technical decision you guys... trying to model natural language on a computer is terrible programming in and of itself
|
![]() |
|
Powerful Two-Hander posted:it's a search complexity problem really, if I search for a value in column X and sort by column Y the whole dataset has to be sorted so a search is always gonna be slow. what I can do though is push the generic "default" search into a different procedure that doesn't need to plan for variables so would return a lot quicker (there is still some variation due to per user access permissions though). its a bit messy though because it means having two procedures returning the same data and testing searches to see if they're a "search with no prior" or "search for filter values that are all empty" Search performance would have to be crippling for this to be a good idea surely? Providing 2 paths to return search results is a maintenance headache. If it is just for sorting then do it in the client unless you're paging the results? I just can't think of any situation I would want to cache search results on the server.
|
![]() |
|
TheFluff posted:trying to model natural language on a computer is terrible programming in and of itself just be grateful nobody decided to just throw tensorflow at it as part of the Unicode committee
|
![]() |
|
Aramoro posted:Search performance would have to be crippling for this to be a good idea surely? Providing 2 paths to return search results is a maintenance headache. If it is just for sorting then do it in the client unless you're paging the results? it's not crippling just laggy, like 4-5 seconds on load and the search is the landing page so every time you go back you've gotta wait for it to rerun your previous search to render the grid content sorting in the client would mean streaming 110k rows from the db to the server, sorting them then throwing 99.9% away to get the first 50 for the page so that's madness. idk the lazy option is use a different query for the basic "no filter" load which will then be more or less instant, caching the last results client side would be better though as they are just json returned to a grid plug in
|
![]() |
|
Powerful Two-Hander posted:it's not crippling just laggy, like 4-5 seconds on load and the search is the landing page so every time you go back you've gotta wait for it to rerun your previous search to render the grid content 4-5 seconds is pretty crippling tbh, that's just under a general users boredom threshold. What would be your strategy for when to burst the cache on the search results? That would be a big bit for me, how out of date can the results be. Your use case here might make caching a good strategy. So the results set is 110K rows and you're just displaying top 50, can you just top the query and reduce that? How interactive is the results set the user can see? Can they sort it etc requiring re-fetching the results? Can they page through the results? If you do cache the results put some indication on the UI at least as to when the query was first run at least.
|
![]() |
|
Aramoro posted:4-5 seconds is pretty crippling tbh, that's just under a general users boredom threshold. it's got it all: sorting, paging filtering and users can have different permissions on what they can see. the permissions are a pain in the rear end because they mean you can't just cache the render result for the basic no filter search currently every query hits the database and though the query is pretty well optimised, it's still got to do a load of work. idk I'll have more of a think about what it makes sen to cache and where good point on the aging of it. I'll add a refresh button titled "blow it out your cache"
|
![]() |
|
idk if this would work in your case, but if it's an internal site you might be able to get away with displaying top 50 in client on their initial search, then streaming the rest to the client in the background so that you don't have to do any caching serverside. 110k rows might be small enough that you can the just do all the sorting etc clientside.
|
![]() |
|
pointsofdata posted:idk if this would work in your case, but if it's an internal site you might be able to get away with displaying top 50 in client on their initial search, then streaming the rest to the client in the background so that you don't have to do any caching serverside. 110k rows might be small enough that you can the just do all the sorting etc clientside. This is kinda what we do. We have user home pages with searches on them, you can have a few of them, and you can defined which columns you see etc. You get your top 25 on each query when you login and you can optionally ask it to show you more or go to the proper search screen if you want to do more. Perhaps a bit of analytics on the user behaviour might help on working out your solution. Like how often do users sort the data, page through it etc. If it's like us then it's never sort, never page because they've already setup their query to show the data they want to display. Depending on the width of the dataset then sorting could definitely be done in the client if you can get the data there sensibly.
|
![]() |
|
i have been splitting a java god class with a god method into smaller pieces. i finished that today. now there are 7 classes with each having one largish method that does a lot of processing. that felt good as is, but then i made one additional change: i had a mehtod with this kind of signature code:
the whole program ran in a single transaction (unnecessarily) and this method ran at the end. the program took about 2 hours to finish when doing a full run. (usually we just process data that has changed since the last run, but even that took close to 10 minutes) after i separated all these methods into their own classes, that also separated the transactions, because the main method that calls all of these methods no longer needed to be @Transactional. that cut down the processing time for this monster method to under an hour. then because i got a hunch, i made a loop in the main method to feed the method 50-element sublists from the original list, each of them in their own separate transaction. that cut the processing time of the whole program doing a full run down to 5 minutes. loving what? the original horrible code was written during the february crunch i have mentioned before. just piling poo poo code on top of old poo poo code because the specs keep changing. then i have had to put out fires and build workarounds for a couple of months because the code performed like poo poo in production. then in may there was another two-week emergency and cleaning out the fallout from that. now i finally had a couple of weeks of relatively chill time to clean up the mess from february and that cuts the worst case processing time from 2 hours to 5 minutes, which just... completely eliminates any performance issues on our end.
|
![]() |
|
the hunch was because i had added progress reporting to the log, and it went processed 100 items, estimated time left 1m 12s processed 150 items, estimated time left 3m 40s processed 175 items, estimated time left 5m 30s the number of processed items kept getting smaller in each time window and the estimate kept growing until it stabilized to about 6 minutes where it stayed for about 30 minutes before it started decreasing.
|
![]() |
|
lmao
|
![]() |
|
i'm searching almost 15 year old C memory here. do execv() and friends do anything to the process space's heap? or is that platform/impl dependent i'm chasing a memory corruption bug that only appears when this library emergency-restarts, and the final call is just a plain old forkless execv. i've not seen this before, and i think it's causing an uninitialized pointer elsewhere to look initialized after the image is loaded and starts running, if that makes sense code:
|
![]() |
|
execv should replace the entire process, the old one will get terminated. all that memory gets reclaimed by the OS. your code looks like windows stuff so im assuming the implementation is broken and dumb as all hell. also how is it "looking" initialized?
|
![]() |
|
CRIP EATIN BREAD posted:execv should replace the entire process, the old one will get terminated. all that memory gets reclaimed by the OS. it's the winCE section, yeah. old product, but it's still got customers and the bug's a P1 ![]() that's what I thought happened but thanks for the confirmation
|
![]() |
|
ctps: wanted to use some third party application that only supports a few auth mechanisms one of which was LDAP, so I wrote up a small server based on apache ds to implement my own in-memory LDAP to fake LDAP bind auth requests and hit the database to check the username/password hash.
|
![]() |
|
Ciaphas posted:it's the winCE section, yeah. old product, but it's still got customers and the bug's a P1 my guess is that the start-up isnt memzeroing poo poo out, and when allocating some struct it's just junk data.
|
![]() |
|
i'm 75% sure all these globals are being used in threaded contexts without enough synch and i'm groaning at having to learn this
|
![]() |
|
just wrap everything with EnterCriticalSection(&CriticalSection); and LeaveCriticalSection(&CriticalSection); hth
|
![]() |
|
Ciaphas posted:i'm searching almost 15 year old C memory here. do execv() and friends do anything to the process space's heap? or is that platform/impl dependent Is the program crashing on that line? If not, is it happening in similar areas in your program? Is the memory being corrupted in the memory of the main process, or the memory of the library? My complete shot in the dark guess is the library might be returning (or consuming) a pointer that's being held on to which shouldn't be.
|
![]() |
|
are you actually doing a fork at some point in that code or are you just using exec to somehow overwrite your own process? because while you should really be doing neither in windows, at least the former is sort of sensible
|
![]() |
|
pseudorandom posted:Is the program crashing on that line? If not, is it happening in similar areas in your program? Turns out the library was being restarted right AFTER it wrote an inventory swap to disk, but before the robot finished the maneuver, and the bug was in the startup inventory reconciliation (missing memset as CRIP EATIN BREAD predicted) So I can fix the reconciliation; the problem is I was never actually able to reproduce the bug--some details were missing in the original report, and try as I might I haven't been able to suss them out enough to even get a 10% repro [there's a race condition involved at the exact moment of the restart, I think]
|
![]() |
|
totally worth it to dig deep and find your best authoritative docs in some MSDN CD set in molding cardboard.
|
![]() |
|
Ciaphas posted:Turns out the library was being restarted right AFTER it wrote an inventory swap to disk, but before the robot finished the maneuver, and the bug was in the startup inventory reconciliation (missing memset as CRIP EATIN BREAD predicted) holy poo poo
|
![]() |
|
Kazinsal posted:are you actually doing a fork at some point in that code or are you just using exec to somehow overwrite your own process? no fork; it's blowing itself away and starting over. the equivalent of ctrl-c and retyping the command, I suppose in this case that execv is happening because a robot axis being controlled by the library (one of the robot axes) reinitialized (power loss), and I guess restarting the library was the solution they went with
|
![]() |
|
I've not encountered this professionally, somehow, so I'll just ask when you've got a bug like this that you think you've put in enough defensive code to prevent relapse, BUT for whatever reason you can't reproduce the original bug in the first place, how is my time best spent going forward - find the drat bug come hell or high water, doesn't matter how unlikely it is to produce in the Real World - code defensively, put in traces, keep an eye out, and hope like hell it doesn't happen again while I get some other work done I guess this is more a 'depends on your manager' question, huh
|
![]() |
|
i'm just gonna assume that thing is using named shared memory as well. sorry for your loss
|
![]() |
|
CRIP EATIN BREAD posted:i'm just gonna assume that thing is using named shared memory as well. yep. it has an option to use a file for inventory instead but by default it heads for either /dev/slram or /dev/nvram, whichever is available, and that was the case here the best part is, when it's using a file, it communicates that to the rest of the system over IP, and I would have seen the contents in the submitted bug logs. Here, because it went through shared memory, and that wasn't logged because of local space constraintt, I have to guess
|
![]() |
|
if you put enough defensive poo poo in, just write to a log somewhere. i haven't touched wince in forever, can you dump a core on failure and then debug it later? if so, enable that. my boss also trusts me to do whatever necessary to get poo poo done (no gods, no managers here), but if it were me i'd just set up something that dumps the core and keeps the last X around.
|
![]() |
|
Ciaphas posted:yep. it has an option to use a file for inventory instead but by default it heads for either /dev/slram or /dev/nvram, whichever is available, and that was the case here lol called it. i think you are me in some weird time loop offset a decade in the past. hello, my past self.
|
![]() |
|
CRIP EATIN BREAD posted:lol called it. i think you are me in some weird time loop offset a decade in the past. hello, my past self. let me know what the winning lotto numbers are for colorado in july, will ya
|
![]() |
|
Ciaphas posted:I've not encountered this professionally, somehow, so I'll just ask option 2. if option 2 isnt viable, then thread title
|
![]() |
|
unless its software where it actually matters, like say you work on mcas
|
![]() |
|
Ciaphas posted:I've not encountered this professionally, somehow, so I'll just ask depends on this impact of the bug and the value of the work you could otherwise be getting done assess the potential impact of the bug, talk to your manager about it and ask what they want you to do. that's literally their job: to translate between engineering and business concerns e: if they tell you to leave it and work on other poo poo, get it in writing, especially if the impact of the bug is bad
|
![]() |
|
Arcsech posted:depends on this impact of the bug and the value of the work you could otherwise be getting done inventory corruption (in this case, the inventory showing a single tape both in the robot picker head AND in a magazine slot, despite the robot being empty & available for moves) bugs are always P1 BUT afaict the picker head inventory contents don't actually matter to the customer that the inventory is being reported to, because a tape being in the robot implies a move hasn't finished, and inventories taken while a move is incomplete are considered suspect. (In fact, other versions of the product don't even report the robot inventory for that reason--just the real storage slots--so that might be enough cause to move on all itself, now I think about it) Ciaphas fucked around with this message at 22:26 on Jun 18, 2019 |
![]() |
|
Ciaphas posted:I've not encountered this professionally, somehow, so I'll just ask as someone who used to have to write software to interface to incredibly unreliable lab equipment option b is sometimes the only solution
|
![]() |
|
![]()
|
# ? Feb 17, 2025 21:11 |
|
oops did that thing where i ramble mostly to myself in a post well, useful notes for tomorrow at least ![]()
|
![]() |