Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
suffix
Jul 27, 2013

Wheeee!

MALE SHOEGAZE posted:

so i'm working on generating interrupts for my emulator now. i can implement it synchronously and i'm sure it will work fine, but i'm very tempted to run the cpu in its own thread and send interrupts to it. this seems to better match how the machine would actually work. am i right here or am i just making excuses to play with futures?

from what i've heard - although having a separate cpu thread that gets interrupts and accesses peripherals would be a reasonable design in theory, in an emulator you eventually end up with specific timing requirements to match the real hardware, like hsync interrupts that must happen with cycle perfect timing interlaced with the screen drawing, so it's easier to do everythin in one thread

Adbot
ADBOT LOVES YOU

suffix
Jul 27, 2013

Wheeee!

gonadic io posted:

poo poo i've just had a great idea. make my own version of try! that automatically annotates the error with standard stack-tracey stuff. failure has a bunch of macros but i didn't see one that did this. too bad that the ? syntax is special and you can't override it with a trait or whatever like you can with + etc

error-chain does this when you do chain_err, i'm pretty sure
from what i can tell error-chain will be deprecated and failure is the new hot stuff but i'm not looking forward to switching, failure looks pretty anemic in comparison

suffix
Jul 27, 2013

Wheeee!

meatpotato posted:

wondering if i can get some feedback on a recent phone interview

for the interview i wrote a moving average class.

it has a circular buffer where (assuming a full buffer) the oldest sample is overwritten as a new sample is written in:

code:
    void add_sample(double sample)
    {
        m_sum -= m_samples[m_write];
        m_sum += sample;
        m_samples[m_write++] = sample;
        m_write %= m_samples.size();
    }
    
    double get_average() const
    {
        return m_sum / m_samples.size();
    }
it seems to work alright

the interviewer also asked how to return the minimum-valued sample in the averaging window in < lg(n) time. I suggested keeping a binary search tree alongside the circular buffer and performing the same deletes/overwrites to it along with the array.

i'm not sure if that's a good answer now since i just read that "classic" binary search trees can't have duplicates. the interviewer also seemed to suggest that only a binary search tree was necessary to solve the problem (not both a buffer and tree), but i can't seem to figure out how that would work.

thoughts on the above? nothing like interviews to make you feel dumb

decent answers imo

you could keep a range minimum query table updated with ln(n) time per insertion, not too hard to code in c

for o(1):
keep a double-ended queue of (position, value) pairs that will have two invariants
- from oldest to newest value
- from lowest to highest value
when you insert a new sample
- pop the oldest sample off the start if it's too old
- pop all values bigger than this value off the end - they will drop off before the new value, so they can never be the minimum
- insert the new value

id visualize it by considering the two cases of always-increasing and always-decreasing samples -
for always-decreasing we can just throw away the old samples, for always-increasing we need to keep them stacked away until it's their time to be the minimum

suffix
Jul 27, 2013

Wheeee!

love the key feature of skein

suffix
Jul 27, 2013

Wheeee!

redleader posted:

i've heard people say poo poo like "history should be immutable because if i'm chasing a bug i need to ~understand the thought process~ that the original developer went through. if they rewrite history, i won't be able to read the total history to ~understand~ how the bug was introduced". you haven't ever done or needed to do this, gently caress you

i have definitely been annoyed before irl at giant squashed "add feature x" commits that touch 300 files, because i was trying to figure out why line n changed, and it was very likely a "oh bar needs to be baz now because the frobnicator failed when i ran the tests" commit originally
remove the wip garbage commits if you must but ffs merge don't squash and rebase

suffix
Jul 27, 2013

Wheeee!
207 multi-status lol

the application has to handle 5xx errors anyway because they could come from a proxy or web server code, so "oh we always return 200" apis is a warning sign to double check error handling off the happy path imo

but i've returned some dirty 200s in my time as a "please forward this to the application instead of raising an exception in the http stack" hack

suffix
Jul 27, 2013

Wheeee!

VikingofRock posted:

I'm actually fine with that. In the context of paths, it's pretty clear that / doesn't refer to division, so overloading it adds convenience without hurting readability (personally I'd say it improves readability). Also I think overloading / for paths is pretty common in languages with operator overloading; for example python's pathlib does it too.

it's terrible in python as well, maybe worse since python code tends to be less overload-crazy in general



your non-async version is technically 'better' here because it can run all the someAsyncFunction calls in parallel, which may or may not matter more than code clarity
i would expect something maybe like
code:
var promises = items.map(item => someAsyncFunction(item)
    .then(result => ({a: result, b: item.someProperty})));
return Promise.all(promises);

suffix
Jul 27, 2013

Wheeee!

DONT THREAD ON ME posted:

hey ya'll, I havent been posting here in a bit. Here's a relatively positive EN for those interested. Semi programming related.

https://forums.somethingawful.com/showthread.php?threadid=3846735&pagenumber=1213&perpage=40#post489294687

best of luck op, ull always be an honorary terrible programmer here

suffix
Jul 27, 2013

Wheeee!

Arcsech posted:

isnt mysql slightly faster if you're basically just using it as a key-value store?

but if you want a key-value store just use redis instead

redis is a cache with some data structures not a db imo

suffix
Jul 27, 2013

Wheeee!
ctps i used to think yaml was an overcomplicated, unpredictable and dangerous file format
but now i'm learning about anchors and merge keys and i'm realizing it is even worse than i thought

suffix
Jul 27, 2013

Wheeee!

Blinkz0rz posted:

spring has these really great concepts called interceptors and filters that can be applied before a request hits your controller. they seem like an ideal place to do hmac validation but apparently HttpServletRequest doesn't cache the request body in any way so once you read it it's gone forever and you can't reassign the body back into the request in any way

so now i'm making a hmac validation utility and calling it from the first line of all my controllers rather than doing it the way i want

if they're like jetty filters you can make a new request that wraps the old one but with getReader etc. overriden and pass that down the chain

suffix
Jul 27, 2013

Wheeee!

Sagebrush posted:

i'm working on a project that needs to be able to detect the presence of one specific object in a video stream. imagine like jian-yang's hot dog/not hot dog software, essentially, but maybe even simpler because i'm looking for literally the same object over and over again instead of a general category of hot dogs. however, the object has a fairly complex shape and i need to be able to detect it from any angle.

i feel like this can be done fairly reliably today on desktop hardware with a well-trained neural network, but I don't know anything about what software i should be looking into. anyone got any names?

probably a small python script with keras or something? https://developers.google.com/machine-learning/practica/image-classification/

i haven't used the fast.ai library but from their posts they try to make it very easy to get something working with sensible defaults https://yashuseth.blog/2018/03/05/hotdog-or-not-hotdog-image-classification-in-python-using-fastai/

suffix
Jul 27, 2013

Wheeee!

abigserve posted:

maybe it's unreasonable to write your own messaging system

yes
kafka is good if you need a simple message log that actually saves messages and it plugs into everything and there's libraries for your language
it gives understandable but strong guarantees and mostly keeps them
if you don't really care about your messages you can use a redis queue

suffix
Jul 27, 2013

Wheeee!
i've been solving the advent of code challenges in both python and rust as a little experiment
the rust solutions roughly averaged take me 1.7x as long to write and run 10x as fast

the moral is probably to write quick one-off programs in python unless theyre going to run for multiple hours at least
the numbers matched my intuition quite well, i do feel more productive in python than in my main typed languages but it's nice to confirm it

suffix
Jul 27, 2013

Wheeee!
tbf if it was running real queries against postgresql i wouldn't call it a 'unit test'

the approaches i've seen is either just mocking the call completely or running queries against a in-process database like sqlite and h2
but you'll need integration tests to catch query errors

suffix
Jul 27, 2013

Wheeee!

gonadic io posted:

It's chromium so :shrug:

that looks familiar so i guess i've 'used' it in the sense that our google kubernetes engine uses it for the node pool

not something you'd notice except i logged into a node to run strace and couldn't find yum or apt

suffix
Jul 27, 2013

Wheeee!
ctps kotlin is quite nice but gradle files and the android documentation isn't

suffix
Jul 27, 2013

Wheeee!

Krankenstyle posted:

uggghhhhh

FileAccess (in file.py) needs to know about and import abstract base class Item (in items.py), of which exists multiple subclasses (in a.py, b.py, c.py). The application only knows about the base class, which instantiates the subclasses based on various conditions, so items.py imports a/b/c.py. One of the subclasses wants to use FileAccess.

how the gently caress do i make python not yell at me for circular imports?

what jabor said but a lazy (hah) way is to defer your imports to the functions that use them so e.g. a.py doesn't get loaded until Item needs to make an A

suffix
Jul 27, 2013

Wheeee!

Finster Dexter posted:

okay you guys that have k8s, are your clusters fiddly as hell? Like, our clusters (including prod) are unstable and the CEO constantly rips on the CTO because of k8s with stuff like "i think all our problems are because of k8s, etc. etc."

on gcp, ime kubernetes itself is stable and reliable - even the persistent storage stuff which was a surprise to me
otoh nginx ingress, google load balancers and the kubernetes network routing are all untrustworthy and liable to drop trafific,
there's churn in the cluster with upscaling/downscaling, upgrading etc.
and of course our own services manage to misbehave and get evicted, or on rare occasions starve other more critical services

overall it has simplified ops but it's basically chaos monkey, you'll have a bad time if you don't have redundancy and retry logic

suffix
Jul 27, 2013

Wheeee!

Finster Dexter posted:

WHAT WHAT WHAT??

Is this documented anywhere... this seems like 100% a deal-breaker, imo. Our system is a giant piece of poo poo built by Russian contractors that is tightly coupled with ultimate trust in NATS as a message queue, and if traffic is just lost, that could explain a LOT of issues we've been having.

If this is a documented known issue, then we need to get off it ASAP

packet loss is a possibliity in any network so lol if you're not confirming delivery
in some limited testing we saw more than there reasonably should be, especially when pods went up and down

no idea about the cause but i figured some combination of endpoint updates being slow and stuff like
https://tech.xing.com/a-reason-for-unexplained-connection-timeouts-on-kubernetes-docker-abd041cf7e02
i don't know if that one has been fixed or not

we're using tcp connections so mostly dropped packages would be retried and just result in some latency jitter but some times we had connection timeouts
all stuff that should show up in your logs and metrics

everything should be retried in any case but just in case we try keep client-facing stuff mostly static, e.g. less aggressive autoscaling so we have a fixed set of pods

suffix fucked around with this message at 18:31 on Mar 13, 2019

suffix
Jul 27, 2013

Wheeee!

Corla Plankun posted:

how do i come up with tasks for a junior Dev to work on? i feel like I'm letting them down by not having work for them but also I'm pretty busy with my own poo poo though?

go through every issue in your issue tracker, and for each one you look at the description, look at them, and sigh and sadly shake your head

suffix
Jul 27, 2013

Wheeee!
maybe with the help of computers and mongodb we could start some kind of "sharing" economy

suffix
Jul 27, 2013

Wheeee!
actually the worst thing about sql is "delete from table"

the second thing is their cobol syndrome (OFFSET start { ROW | ROWS } FETCH { FIRST | NEXT } [ count ] { ROW | ROWS } ONLY)
and the third probably something something sql injection

suffix
Jul 27, 2013

Wheeee!

that all sounds like the normal way to do it
you should also save away which packages you are using so you can recreate the venv later - you don't check the venv directory into source control

something like
pip freeze > requirements.txt

pipenv does that part better by keeping a Pipenv and lock file for you, but i wouldn't worry about it if you're happy with your current workflow

suffix
Jul 27, 2013

Wheeee!
ctps after considering a number of options for my use case, mongodb looks like the closest fit

that probably means i should reevaluate my use case

suffix
Jul 27, 2013

Wheeee!

gonadic io posted:

postgres's json mode?

it's more about operation than raw capabilities really

happy for any suggestions, requirements are:

- load is read heavy, mostly k/v, but ad-hoc queries for debugging is a nice to have
- deploy on kubernetes (arguably dumb but simplifies stuff since most our infrastructure is built around it)
- high availability without babysitting
- larger than memory dataset but much of it is cold
- preferably easy scaling
- java and python client support, not super obscure

we can tolerate relaxed consistency + durability

postgresql can probably do ha but i don't know off hand how to set it up for fixed nodes, let alone in kubernetes
my intuition is that a db with built-in failover should handle the hostile kubernetes environment better than a secondary+sentinel system

there's no shortage of helm charts for every database, but ime even the official chart repository has charts with broken metrics and rickety replication that show they haven't been used in anger
it's hard to know which solutions to trust

suffix
Jul 27, 2013

Wheeee!
maybe ill use a kubernetes operator to provision and teardown a clod db

suffix
Jul 27, 2013

Wheeee!
pair programming sessions are if nothing else good for disabusing juniors of the notion that you're a senior because you have all the apis memorized
also shows effective googling, like use the official docs not some medium blogspam, and you'll probably get some "wow, i didn't know the ide could do that" comments

people talk about benefits to code quality but basic tricks and shortcuts are undershared, much like bum wiping techniques

suffix
Jul 27, 2013

Wheeee!

Sapozhnik posted:

Imagine thinking you need k8s

old man yells at cloud.txt

suffix
Jul 27, 2013

Wheeee!
we removed a lot of foreign keys because they were interfering with online schema migrations, iirc

i should check some time how hosed our data is

suffix
Jul 27, 2013

Wheeee!
saw this today
code:
inline void crash() { *((volatile unsigned *) NULL) = 0; }
feels like this is just asking the compiler to smite you

suffix
Jul 27, 2013

Wheeee!
keyword only parameters can be good but positional only is a minuscule edge case so it's probably safe to just forget it exists

suffix
Jul 27, 2013

Wheeee!
yolo memcache everything, can't have slow queries if you never go to the database

suffix
Jul 27, 2013

Wheeee!

Shaman Linavi posted:

just finished my time on the bug team and lol i don't even know what the team before us did. we closed 95 bugs (all highest priority of course) and already have 130 in the backlog with a big release coming up next weekend. fixed crappo that was going to get us sued real nice in a few states. glad im not on the rotation again until september*

*im going to be on it forever because turnover

what kind of system has a rotating bug team? you don't pass bugs back to the team that wrote the component?

suffix
Jul 27, 2013

Wheeee!
you could probably do it in a simpler and better way than loading it into sqlite and running the queries you need. but you could also do a lot worse

i'm sure i've seen cli tools that basically just load a cvs file into sqlite and gives you the prompt to query the data

suffix
Jul 27, 2013

Wheeee!

animist posted:

are there any nosql things with anything approaching reasonable transactions

what's a reasonable transaction to you? e.g. foundationdb, dynamodb and firestore all have transactions, but with a somewhat limited working set
good for "update these two things together", not good for "replace the entire table atomically"

suffix
Jul 27, 2013

Wheeee!

Krankenstyle posted:

is nginx the one where you write the config in some C subset? cause i worked with that one for a while and it fuckin sucks and it would be a good thing if that kind of configuration managementwas forever forbidden everywhere

probably varnish, the configuration is compiled and you can actually inline C

suffix
Jul 27, 2013

Wheeee!

Share Bear posted:

cool so if youre doing pointers and simple memory addressing to teach a new person poo poo like that, how is it improved over C

how does it make it more approachable?

it would give you the ability to say "of course in real life we'd just use the standard library implementation" and to do advanced poo poo without reimplementing vector and map every time

it's not entirely clear to me if you're talking about bare-metal "this is how the linux kernel does it", in which case c or knuth-style assembly is good,
or if you mean data structures w algorithms, like heaps/b-trees/graphs etc, which python works perfectly fine for imo

quote:

better errors?

lol no

suffix
Jul 27, 2013

Wheeee!
i've only just started on the spring boot docs and i've already seen several mentions of things being automatically set up if they're available on the classpath

getting a bit scared

Adbot
ADBOT LOVES YOU

suffix
Jul 27, 2013

Wheeee!

CMYK BLYAT! posted:

Helm is a trash fire. their literal official guidance in this scenario is "just stick YAML comments in front of all your template lines because then it won't be invalid when it renders", ignoring that this often makes other things invalid and gets really fucky when you have several levels of nested loops and includes

i thought we'd use helm for some months until a better alternative got popular but it's been years now please send help

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply