|
Dumb Lowtax posted:If I just have a regular array of 70 floats, and I look up entry #69, isn't the CPU likely to pull a bit of adjacent memory into cache as well due to the expectation that my memory accesses will have some kind of coherency? Don't reads/writes tend to happen in bigger chunks than a single float for that reason? so, first of all, for a regular hash table (which you have adamantly insisted you are talking about), we're using some key, which we've hashed, to find a single object in the hash map. so we don't care about bucket #68, or bucket #70, we care about bucket #69. and we care about iterating through the entries in bucket #69 very quickly. so we would want our cache to be filled with bucket #69. your proposed algorithm might make looking up bucket #68 slightly quicker at the expense of torpedoing bucket #69, because now all our bucket #69 accesses are very far apart. but we have no reason to care about bucket #68, as in a normal hash table, hashes should be roughly randomly distributed to keep the load factor of the table down. now, i do not believe you are talking about a regular hash table. let's imagine we're doing some basic spatial hashing thing where we care about a list of objects inside each bucket as candidates for a search, rather than an exact key match. your algorithm still sucks. it is made worse by the fact that your tables are uneven. say we have three objects in bucket #69, and one in bucket #68. by splitting the objects out in bucket #69 into three separate tables, when we ask to load table 3 for bucket #69, we're going to throw out table 1 for bucket #68 and replace it with mostly empty entries that aren't relevant for us! so, most of the cache is going to table entries which are empty. congrats!
|
# ? Nov 14, 2019 09:57 |
|
|
# ? Jun 15, 2024 13:38 |
|
also, i just want to point out that your suggestion to improve cache efficiency is to stride out your accesses over a large distance, and my only response to this is "lol"
|
# ? Nov 14, 2019 10:06 |
|
Comedy option: use machine learning to predict the next memory access location and prefetch for you. (... somebody has done this haven't they) E: Sorry about jumping in an explaining spatial hashing, I hadn't quite grasped what weird variant was going on here.
|
# ? Nov 14, 2019 10:12 |
|
Linear Zoetrope posted:Comedy option: use machine learning to predict the next memory access location and prefetch for you. depending on how vague your definitions are, some modern cpus already work this way, with perceptron-based branch predictors
|
# ? Nov 14, 2019 10:17 |
|
Linear Zoetrope posted:Caches don't Ah, I did not know that effect of pulling in adjacent memory at once was so narrow in its window. That makes methods like Z order curves considerably less appealing. There's a few blog posts about learning Z order and show off Morton coding (including 3D) that play up its benefits for memory locality a little too much. They make it sound like a vital 3D industry technique when really might have just been showing off trivia. I think that's my confusion. Linear Zoetrope posted:Even in the best (worst?) case where you have a collision in every bucket to check, allocate a static amount of memory for objects per bucket, and have every one of those slots filled, you can choose exactly one of left-right and top-down cache coherency depending on whether your grid is row or column major. Yeah, you're describing what the aforementioned Z order curves are. Coherency would not depend on row / column major because that's not the order we've stored the grid in. Linear Zoetrope posted:Do you actually know this is a performance problem for your use case or is this a thought experiment/self-imposed engineering challenge? No, I wrongly inferred ahead of time that it would be a major performance problem based on the existence of said blog posts, which I presumed meant other people had hit similar performance issues that were only solved by using Z order curves, and that they inevitably become a must-have when your acceleration structure is a spatial hashing grid. Apparently that's not the case at all, and having looked up neighboring cells recently is not in fact an important factor in whether the current cell you want to query has been pre-fetched. If that's true, then I might as well use random hashing with open addressing and be done with it. Just a simple map of integer XYZ triples onto contiguous Vectors of the data that lives in each cell. Thanks for using the term "spatial hashing" the way I've seen it used, and demonstrating and defining it for everyone. It's a cool technique with an unfortunate name. Linear Zoetrope posted:I don't think spatial hashing is really all that common anymore, it was more common a decade ago but I think only in indie spaces because it's better performance than brute force but comparatively easier to implement than some of the tree-based methods. It's also, in theory, marginally faster than rebuilding an octree under the right conditions (usually when there are very few dynamic objects that are generally small enough to fit in exactly one bucket in almost all frames). As late as 2012 I did see the technique used in the industry, not for indie games but a major VFX studio, when we were implementing our new simulation engine. Happy Thread fucked around with this message at 11:03 on Nov 14, 2019 |
# ? Nov 14, 2019 10:40 |
|
As a programmer working at a company, how do I estimate how much revenue I produce personally for the company? This is a big question, so let me add context. I work on a group of products that together make about $400m in revenue per year from subscription, and there are maybe about 200 employees - developers, UX, PMs, managers etc total working on it. I've been a developer on this product for a year and a half and I got this job fresh out of college. Typically, my workflow is that the PMs make a roadmap of features they want in the next release, and every quarter the PMs, POs, UX and individual developers like me work out features for each team to do. I am assigned features to work on, and then do the typical developer stuff - write user stories with estimates, put them in sprints, implement them and have QA test them. Really the main "value" here is in the features proposed by the PMs, and I add value by implementing the feature on time to spec. So then, we could say the revenue is some percentage of the revenue generated by that feature. Except that itself is a pretty fuzzy definition. Users aren't buying the feature, they're buying the product, and the revenue from that feature is the number of subscribers we either retain or add because of that feature. Right? Am I wrong in that assessment? So I asked the PMs "do you know how many subscribers you'll get from this feature?" and they say that they don't know; apparently it's often hard to translate a specific feature into a prediction about subscribers added. They just follow from what users have asked for and ongoing trends in the industry. So how do I know how much revenue I've actually produced? How do I know how much I am actually "worth" to the company? My salary doesn't really say that, because the salary is more a reflection of supply and demand for programmers in the greater market. I'm sorry if this doesn't quite fit the thread, I could ask somewhere else if necessary. I'm just a bit surprised by the fact that I've been working at this company for a year and a half and I still don't even know if I'm worth what I get paid. Edit: I should also note that I have a calibration every 6 months or so, and I review my work done with my manager. He seems to be happy enough to promote me, so I'm certainly doing well, but the actual revenue I generate is never a topic. America Inc. fucked around with this message at 02:40 on Nov 17, 2019 |
# ? Nov 17, 2019 02:29 |
|
It's fine to not be in a revenue generating activity. Most revenue producing programmers work for agencies, which is poo poo work. If you want to make the case that you're worth the expense, read the book The Real Business Of IT.
|
# ? Nov 17, 2019 03:59 |
|
rt4 posted:If you want to make the case that you're worth the expense, read the book The Real Business Of IT.
|
# ? Nov 17, 2019 06:14 |
|
LookingGodIntheEye posted:As a programmer working at a company, how do I estimate how much revenue I produce personally for the company? If this is for some internal survey, it's $1m If this is for self worth, don't judge yourself by how much revenue something pulls in, lest you fall into the MBA trap.
|
# ? Nov 17, 2019 07:03 |
|
Dumb Lowtax posted:I wrongly inferred ahead of time that it would be a major performance problem I was going to give you the usual “measure first” lecture, but it seems you don’t even have a performance problem yet... You are very far away from even considering harebrained bespoke hashing solutions.
|
# ? Nov 17, 2019 08:14 |
|
Adhemar posted:I was going to give you the usual “measure first” lecture, but it seems you don’t even have a performance problem yet... I wasn't. I wasn't considering anything. I was employing a made up example to better articulate a question. Jesus. Like, are you really going to make me say it? Ok, have you morning coffee this time and look again, because I did not in fact post some thread proclaiming "eureka!!!" I have found a better way to hash!" or whatever you are picturing, nor did I even once say it was a good method or idea. I asked: If it is, then tell me the name of it -- because in that case it's surely already some well known variant -- and if it's not then help me find the part I'm still missing, ie. help me to notice the counterexample so that I can understand hashing. There's a reason I ended my original question with "what am I missing?" and posted it with a question mark. There's a reason I filed it away at the end of the stupid programming questions that don't even deserve a thread thread. The replies I got really have an odd tone considering. I'm having to defend myself from accusations of crankery twice because I honestly asked any question that requires setup for an example/counterexample? I fully understood that I'm not going to discover a better variant of hashing, something that a million people have already looked at and already had a chance to find. I get that, I never acted otherwise, and I just asked for the details. What some of you need to get is that if someone is actively taking steps to confront their own misconceptions and improve understanding, that does not make them out of line or needing to be treated like a lovely practitioner. And you know what, my curious questions and I are gaining on some of you each month. Clearly if someone were novice *enough* here and asking far more basic questions full of big misconceptions you all would be taking pride in holding a newcomer's hand as they find programming. I should take it as a complement that some of you don't. Maybe when it's a closer match you can see a little of your former knowledge level staring back at you and it scares you. Happy Thread fucked around with this message at 10:52 on Nov 17, 2019 |
# ? Nov 17, 2019 10:35 |
|
Settle down. The lesson is "don't optimize for a problem you don't even have" not "ur dum lol"
|
# ? Nov 17, 2019 15:53 |
|
I dunno, the guy coming in out of nowhere three days later to talk poo poo seems a bit obnoxious.
|
# ? Nov 17, 2019 15:55 |
|
Volmarias posted:Settle down. The lesson is "don't optimize for a problem you don't even have" not "ur dum lol" Again. I did not optimize for anything. I asked how hashing and locality work. I asked what my understanding of it at the time was missing because it would imply everyone doing it differently than the way they clearly do. I didn't invite or need a lesson, I needed an answer if anyone cared to.
|
# ? Nov 17, 2019 21:31 |
|
To be fair, it's super common for people to ask the exact same sort of questions and actually be prematurely optimizing or thinking they've created some unique and amazing data structure that has never existed before.
|
# ? Nov 17, 2019 22:45 |
|
If I had a piece of advice for you, it's to work a bit more on your communication. The description of the problem and your proposed solution took multiple people to tease out quite painfully, and you still don't seem to understand what hash collisions are despite being consistently adamant that you understand them. Stop using fancy words, explain things in much simpler terms, and don't double down when someone tries to clarify things for you.
|
# ? Nov 18, 2019 06:25 |
|
Volmarias posted:Settle down. The lesson is "don't optimize for a problem you don't even have" not "ur dum lol" Yeah exactly this. Jabor posted:I dunno, the guy coming in out of nowhere three days later to talk poo poo seems a bit obnoxious. I just quoted DL’s most recent post, after catching up on this thread (and admittedly getting a little frustrated by the whole exchange). I wasn’t looking at dates and didn’t know that much time had passed. Apologies, I didn’t mean to stir up more poo poo. I’ve just seen a lot of programmers optimize prematurely. That includes myself as a young whipper snapper.
|
# ? Nov 18, 2019 08:02 |
|
I'm trying to roll my own pubsub system in C#, mostly because I thought it'd be pretty straightforward. But making it clean (i.e. not having a million little subscribe/unsubscribe functions for each individual event type) is proving a little trickier than I'd thought it'd be. I have this class:code:
|
# ? Nov 20, 2019 19:32 |
|
Question about flow, I want to write a program that will do the following (in Python preferably): 1. Run some calculations in a loop based on timedelta/tick rate i.e. its deterministic (?) so if it takes 2 seconds to do a loop then 2 seconds of changes should occur and I'll have some clock to rate limit the loops, kind of like a game loop tick rate. 2. While the loop runs various calculations, be able to query against the application to get data. For example, say the loop is calculating the position of something moving so you have x,y coords updating every loop. I would want to be able to get the current position via some query . I'd like to do this via an API like Flask. Whats unclear to me is whether I can do this in Flask and have some complex background thread running doing all the updates while I create the API endpoints to get information, or if I need to have a task sending the data to some intermediate place like a database for storage, then a separate app running for the API side against the database.
|
# ? Nov 20, 2019 19:33 |
|
Sylink posted:Question about flow, I want to write a program that will do the following (in Python preferably): Deterministic means that if you run the same thing with the same input you get the same result. The problem is that you're having up account for time variance, so that wouldn't necessarily be deterministic if you have large lags for some reason causing you to miss some input. It also depends on whether you want to store the raw data and rerun it for a reason. If this is a trivial 1:1 server - experiment relationship and you're ok with things disappearing forever if the server restarts, you don't need a data layer like a database. You also don't need two separate apps, as long as your threads work relatively well; you should be able to do this with some simple synchronization E.g. Python code:
As long as none of your locked sections have a way to run forever and thus block the other, this should be enough. Volmarias fucked around with this message at 20:54 on Nov 20, 2019 |
# ? Nov 20, 2019 20:51 |
|
TooMuchAbstraction posted:I'm trying to roll my own pubsub system in C#, mostly because I thought it'd be pretty straightforward. But making it clean (i.e. not having a million little subscribe/unsubscribe functions for each individual event type) is proving a little trickier than I'd thought it'd be. I have this class: 1) You should probably ask this in the .Net thread 2) I haven't actually written C# in years, so I can't see why that wouldn't work, but If T is already constrained to PubsubEvent, then why not just write subscribers[typ] = new SubscriberSet<T>(); 3) Interface names should start with I, i.e. public interface IPubsubEvent {}. This is purely a convention, and has no effect on the underlying code, but it's so ingrained in C# and .Net that I spent a few minutes seeing if there was some kind of weird issue where the compiler might not like doing whatever with a derived class before I realized PubsubEvent wasn't even a class.
|
# ? Nov 20, 2019 20:53 |
|
I have a WPF application I made using MVVM standards as a learning exercise. I also have an API that I made that communicates with a MongoDB server. Is there a design convention for having the WPF app consume the API? I was basically just gonna make model classes for the different types of objects on my server, a model for the response data, and then just make a class with the logic for grabbing data. Is that reasonable? I ask because this learning stuff is on my resume and I want to make sure the "architecture" is at least somewhat thought out.
|
# ? Nov 20, 2019 22:10 |
|
Jethro posted:1) You should probably ask this in the .Net thread Thanks, will do! For your second question though, the compiler doesn't like that either; says Cannot implicitly convert PubSub.SubscriberSet<T> to PubSub.SubscriberSet<PubsubEvent>.
|
# ? Nov 20, 2019 22:16 |
|
When someone says they're buying something for $30 shipped plus exchange fees, does that mean person receiving the money and shipping the item are expected to pay the exchange fees and shipping?
|
# ? Nov 20, 2019 22:26 |
|
exchange fees no, shipping yes e: to clarify: $30 shipped means shipping included, plus exchange fees would mean the exchange fees are in addition to the $30
|
# ? Nov 20, 2019 22:33 |
|
TooMuchAbstraction posted:Thanks, will do! For your second question though, the compiler doesn't like that either; says Cannot implicitly convert PubSub.SubscriberSet<T> to PubSub.SubscriberSet<PubsubEvent>. What you're running into is called generic variance. It's easiest to explain with an example: - If you have a List<Derived>, you can't pass it to a function that wants a List<Base>, because the function might try to insert a Base into the list, which the List<Derived> cannot allow. - If you have a List<Base>, you can't pass it to a function that wants a List<Derived>, because it might read a value from the list and get something that isn't a Derived. These two properties mean that a List is invariant - a List<T> can only be a List<T>, and can't have any other type parameter. Compare that to Action: - If you have an Action<Derived>, you can't pass it to a function that wants an Action<Base>, because that function might invoke the Action with something that isn't a Derived. - But note that there's no problem with passing an Action<Base> to something that wants an Action<Derived> - there's no way for the code to "read out" something from the Action and get a Base where it expected a Derived. So Action is contravariant. Your main issue is that you're trying to put Action<T> into a SubscriberSet<PubsubEvent> - which cannot be allowed, because those Actions cannot actually accept arbitrary PubsubEvents. What you really want is a mapping from Type to SubscriberSet<that same type>. Unfortunately, there's no easy way to express that constraint in the type system. Usually, when you have a complex constraint like this that you can't express in the type system, you use casts to tell the compiler that your constraint is true. Ideally you limit that to a small part of your code, and maintain proper type safety outside of it. So you could do something like: code:
|
# ? Nov 21, 2019 00:34 |
|
Thanks for the detailed explanation! That makes sense, and while I was hoping to avoid too many casts, I guess it's not so easy to avoid in this situation. At least it's not the kind of thing where the loss of type safety should leak into the rest of the codebase; everyone outside of this class should be able to make strong assertions about types.
|
# ? Nov 21, 2019 01:47 |
|
I want to hook into whatever Windows is doing to populate the Currently Playing OSD thing and get the track/artist info. It seems to be based on the UWP System Media Transport Controls. I'm not really up to speed with whatever MS is doing nowadays with the apis, is it possible to interact with this from a native app somehow?
|
# ? Nov 21, 2019 16:53 |
|
mobby_6kl posted:I want to hook into whatever Windows is doing to populate the Currently Playing OSD thing and get the track/artist info. It seems to be based on the UWP System Media Transport Controls. I'm not really up to speed with whatever MS is doing nowadays with the apis, is it possible to interact with this from a native app somehow?
|
# ? Nov 21, 2019 20:32 |
|
I'm having a problem. It's with a utility called nncase. Name of the actual binary is ncc. But whatever. This: https://github.com/kendryte/nncase What's the best way to seek help? The problem I'm having is the output just doesn't seem to work. I've built three different versions of v0.2 over the past few months hoping the output (a hardware specific neural net model file) will work. It doesn't. I saw the author uploaded a premade flashable binary in the last day. Ie a whole self contained ROM image. I tried that and managed to catch via the UART that it couldn't use the model. I normally use micropython, not the standalone SDK, RTOS SDK, or ArduinoCore because it lets me fiddle around. I thought it was a problem with my models, which were made using tutorials. The v0.2 version of the ncc utility has a couple of modes. compile and infer. Compile converts the tflite model to kmodel and does some kind of retraining because it wants the dataset. infer also wants the dataset, but it pumps out a stack of .bin files to a specified directory. The files are the numpy tensor output of the output layer. I can view that but I can't understand it. They certainly do look like a valid output though. My question is how do I tell the author of the utility that it doesn't work for me in any form on the actual hardware without sounding like a complete idiot?
|
# ? Nov 21, 2019 21:41 |
Describe the exact hardware you have, the exact steps you have taken to attempt to make the software work with the hardware, and provide verbatim any diagnostics you have collected. (Don't paraphrase error messages, copy-paste or screenshot or photograph them. Take videos if you need to.) Make sure to describe your execution environment, i.e. OS version, other general system specs, and what hardware you use to interface with the actual device. Make sure you have tested with a known-good input that should be compatible with your hardware. nielsm fucked around with this message at 21:57 on Nov 21, 2019 |
|
# ? Nov 21, 2019 21:54 |
|
General_Failure posted:My question is how do I tell the author of the utility that it doesn't work for me in any form on the actual hardware without sounding like a complete idiot? lol i regularly pratfall on basic issues in front of professionals, why do you care about an internet rando’s opinion you’re coming from a good place, but don’t worry about looking like an idiot, just try to ask the author a straightforward question with all relevant context that will allow you both to move forward. you’ve done a lot of good triage on your own! for next steps see if you can find the code that emits ‘bad model’ error string and find what it’s checking for, but that can escalate quickly. provide all the steps you’re doing to generate the models, if the output is reasonable you can attach all of it. check any relevant documentation, including the SDK, make sure the tools aren’t printing some obvious error message. yeah, you might be missing a silly cmd line flag and the author will answer you in 5 seconds while scoffing harder than i ever have, but they’ll probably just be happy someone is trying to use their stuff! if your issue is common and you ask it well, it’ll help others get over the same roadblock, or the dev might add documentation to cover this case.
|
# ? Nov 21, 2019 22:03 |
|
If you were to just copy paste what you posted here into a github issue, you wouldn't sound like a complete idiot. You are clearly earnestly trying to make it work. If the author shits on you, that's on them being a dick IMO. (also extremely unlikely in my experience)
|
# ? Nov 21, 2019 22:37 |
|
Lots of good stuff in the replies. Thanks. I just tried the standalone bin on a different bit of hardware. Previously I've been using a Maix Bit. Mostly because of the micro SD card slot. I can say that the models don't work when they are written to flash, or stored on the SD card and loaded. I just tried the same premade bin that was uploaded yesterday on my Grove AI HAT. Same result. Keeping the messed up formatting. code:
code:
|
# ? Nov 21, 2019 23:45 |
|
I think code 2 is #define MP_ENOENT (2) // No such file or directory set here: https://github.com/sipeed/MaixPy/bl...ffs_file.c#L124 e: If you run the program under strace you can probably see what file it's trying to open. (I don't know if that works with python involved) taqueso fucked around with this message at 00:02 on Nov 22, 2019 |
# ? Nov 21, 2019 23:56 |
|
taqueso posted:I think code 2 is #define MP_ENOENT (2) // No such file or directory I'll have to try that. There is an example on my SD card that works. Probably prepared with an early version of ncc. Testing whether a kmodel will load is easy in micropython because I can do it in immediate mode. code:
|
# ? Nov 22, 2019 01:36 |
|
I asked. I got my answer. It only supports the standalone SDK. I guess the micropython environment hasn't caught up.
|
# ? Nov 22, 2019 12:01 |
|
I have a question about code generation. I've been parsing Python using ANTLR and creating my own bytecode instructions out of it similar to what Python itself is doing. I'm doing this without creating an AST. ANTLR now uses the visitor pattern to traverse through the syntax through the statements given in the grammar. I've just been vomiting byte code operations from that. The code to do this does get a little goofy--especially when it comes to fixups for breaks in code for conditionals or loops. I'm generally curious if this becomes more manageable in some way if I switch to an AST. I'm asking instead of just trying it because I'd have to invest quite a bit of time to get to a point where I could even tell. So I'm wondering if there would be benefits for maintenance if it went grammar->visitor->AST->bytecode versus just grammar->visitor->bytecode.
|
# ? Nov 22, 2019 23:05 |
|
Typically, the data structure you want to handle flow control is a flow graph, which is a series of blocks, linked by an entrance and potentially many exits. You can build that from whatever you want. Most people hand-write recursive-descent parsers which will build an AST, but you can also build the flow graph from a generated parse tree if you want, or inline like ANTLR lets you do.
|
# ? Nov 22, 2019 23:30 |
|
|
# ? Jun 15, 2024 13:38 |
|
Suspicious Dish posted:Typically, the data structure you want to handle flow control is a flow graph, which is a series of blocks, linked by an entrance and potentially many exits. You can build that from whatever you want. I do have that already. I didn't mention it because it surprisingly did little to improve readability in the ANTLR visitors. It did help with correctness though. Edit: I guess I should say that I'm asking if doing an intermediate step with an AST is generally smart for a process like this. My impression is the AST-to-bytecode step might be pretty straightforward, but I will still have some gross blocks of code to generate the AST in the first place. Rocko Bonaparte fucked around with this message at 07:07 on Nov 23, 2019 |
# ? Nov 23, 2019 04:23 |