|
Paolomania posted:I wish our interviewing practice was more like a pair-programming session where you sit down with some mild refactoring task and get to talk about how to redesign something and reasoning behind various choices. I feel like you get a feeling for not only how much coding experience they have on some particular platform, but how they think and how they communicate.
|
#
?
Jan 27, 2021 09:57
|
|
|
|
| # ? Apr 26, 2024 15:25 |
|
|
Sagacity posted:This is something that works really well. When I interview someone I take a service with a large codebase, introduce a bug that causes a few tests to fail and tell them "please fix and keep talking to me while you work". I like this. Do you use something proprietary or open source?
|
|
#
?
Jan 27, 2021 14:52
|
|
|
Pedestrian Xing posted:I like this. Do you use something proprietary or open source? 
|
|
#
?
Jan 27, 2021 16:33
|
|
|
|
|
#
?
Jan 28, 2021 22:41
|
|
|
Edit: whoops, moving his to Career Advice thread
Woodsy Owl fucked around with this message at 00:23 on Jan 29, 2021 |
|
#
?
Jan 29, 2021 00:19
|
|
|
holy loving poo poo the component I have been designing and working on for the past year just had a successful run with all the moving parts and I am so happy I want to die thank gently caress I somehow managed to get two mostly-meeting-free days in a row to knock the last kinks out it is so goddamn cathartic to have been able to spend the last week just crunching to deliver code instead of being mired in infinite meetings and not being able to deliver to any timelines because there are meetings forever oh god make them stop For real this is like my first tangible achievement since lockdown started and I cannot begin to describe how happy I am right now goddamn
|
|
#
?
Feb 25, 2021 22:14
|
|
|
ChickenWing posted:holy loving poo poo the component I have been designing and working on for the past year just had a successful run with all the moving parts and I am so happy I want to die I'm celebrating here with you, I'm glad you got to make something good
|
|
#
?
Feb 25, 2021 22:31
|
|
|
CPColin posted:For my entire career, I've had to grapple with the simultaneous feelings thst I'm both overpaid and underpaid. Underpaid for the bullshit we undergo, the stress, the accountability put onto ourselves to do irresponsible things for the sake of future workers
|
|
#
?
Feb 26, 2021 02:57
|
|
|
Am I crazy for thinking that end-to-end test suites are just complete horseshit? In my admittedly short 6 year career I've had to maintain and develop end-to-end tests across a variety of frameworks, and each one just ends up being a stupid section of the codebase that we have to keep maintaining to keep pipelines happy but every time a test fails it's always due to something super finnicky with the automation itself and never because an actual bug was introduced to the application. I'm spending over half of my development time adding these tests for each of the stories I work on since that's part of our requirements and it honestly just feels like a waste of time. Unit tests and Integration tests are great for verifying your logic is working but I've never seen an e2e pipeline that didn't have these issues. This is my first project as a frontend developer (backend is handled by a separate team) and it's honestly been a nightmare, gently caress selenium and cucumber in particular.
|
|
#
?
Feb 26, 2021 05:00
|
|
|
Fano posted:I'm spending over half of my development time adding these tests for each of the stories I work on since that's part of our requirements and it honestly just feels like a waste of time. Pretty much my experience as well. They're all super fragile since everything is referred to by ID, and the whole suite is incredibly slow because every variable (1200+) is public static so nothing can be run in parallel. Then the idiots that put the whole abomination together get to critique our code whenever we write tests.
|
|
#
?
Feb 26, 2021 05:55
|
|
|
Deciding what to test is as important as whether to test at all. Bad tests wind up testing things that are unrelated to your domain or systems, which is where e2e tests tend to be pretty useless. For example, if a machine runs out of disk that doesn't matter except if you're an OS developer or similar. But over in SRE and infrastructure land literally 80%+ of our code is about handling stuff that product / feature engineers don't want to be bothered with (and shouldn't!) because they're all things dealing with computers, not the business domain that delivers the value people want out of the whole project. However, there's definitely times when my end to end tests have caught changes to things we didn't know we relied upon, and they can be useful for exploring a system and discovering your real requirements because your system wasn't engineered as much as it was evolved organically. For example, a timeout in a transaction across several services wasn't really understood and people didn't realize one day that two services that started taking just a little longer but met their own internal SLAs would result in 500 errors in 3 other services. So we had to add in some new requirements and tests guaranteeing that timeout values in the call graph don't sum beyond the previously-unknown transaction boundary. I think of e2e tests as crutches for actually knowing and owning the boundaries, failure modes, and interfaces between decoupled systems. Because really testing in production to me is simply observability which should be the same tooling capabilities that development has for simple local programs ideally with debuggers, profilers, etc.
|
|
#
?
Feb 26, 2021 06:11
|
|
|
Fano posted:Am I crazy for thinking that end-to-end test suites are just complete horseshit? My experience so far has been that end to end tests are only useful for a very narrow/stripped back regression test of core functionality to give quick feedback on anything that's been accidentally broken. Caveat: I've only ever worked at very small companies. I imagine my view would change in a larger organisation, or if I were working on something with substantially more moving parts. So... YMMV.
|
|
#
?
Feb 26, 2021 06:27
|
|
|
froglet posted:No, not crazy. Saying this not as a developer, but as a tester. No, the problems just get worse at larger companies. Having a small set of end to end tests the validate that the application is deployed and running and can execute a few critical paths (think "search for item, add to cart, go through checkout pipeline" in a store app) is fine. Testing any more complex business logic at that level is an exercise in futility and pain.
|
|
#
?
Feb 26, 2021 06:55
|
|
|
Most end-to-end tests are poorly written reproductions of the unit tests in python  Thinking of and writing meaningful tests is much harder than writing app code, but most companies treat QA as “developers not good enough to write real code”. They’ll hire for a job which 90% consists of writing difficult code but accept someone who’s barely technical and can’t really grasp the concept of libraries, let alone the kind of stuff necrobobsledder mentioned. And then when the tests do catch things like timing issues, we all shrug and say “prod doesn’t have that issue. Put a delay in.”I saw the straight up dumbest poo poo with Cucumber/Gherkin. What a doomed idea. All the code problems plus a language barrier.
|
|
#
?
Feb 26, 2021 09:39
|
|
|
New Yorp New Yorp posted:Having a small set of end to end tests the validate that the application is deployed and running and can execute a few critical paths (think "search for item, add to cart, go through checkout pipeline" in a store app) is fine. Testing any more complex business logic at that level is an exercise in futility and pain. Came here to say this. I'm all for having a handful of "smoke tests" that check if login is working and maybe some checks to make sure every layer is talking to each other correctly. Although it's frustrating that most of the services rely on super brittle CSS selectors or worse. We're in the middle of a bit of a rewrite currently to make sure some of our core modules are more easily tested by unit tests mainly so we can avoid adding more e2e testing. I do enjoy using headless browsers to mess around scraping other people's websites. That's just good old fashioned fun.
|
|
#
?
Feb 26, 2021 12:52
|
|
|
New Yorp New Yorp posted:No, the problems just get worse at larger companies. Having a small set of end to end tests the validate that the application is deployed and running and can execute a few critical paths (think "search for item, add to cart, go through checkout pipeline" in a store app) is fine. Testing any more complex business logic at that level is an exercise in futility and pain. 100% it's just worse at scale. Also tends to depreciates the perceived value of all testing in the eyes of some mgmt because they hear eng ignoring breaks.
|
|
#
?
Feb 26, 2021 14:12
|
|
|
How do you distinguish e2e tests from automated QA tests? Or is QA automation a pipe dream doomed to be too brittle forever?
|
|
#
?
Feb 26, 2021 14:50
|
|
|
we have a pretty extensive collection of automated qa tests and its always broken. our testers get so excited whenever the pass rate is over like 60%.
|
|
#
?
Feb 26, 2021 14:52
|
|
|
We have end-to-end tests that pass reliably, to the point we use them to run performance benchmarks on every release. We're not really testing a whole lot of business logic though - just start the app, go through this flow, did you get to the end without crashing and see everything you expected to see? It doesn't replace QA.
|
|
#
?
Feb 26, 2021 15:22
|
|
|
We have a ton of Selenium tests for our little web app and I would say at least 50% of failures that they detect are out of my control as a dev. Since we make software for financial companies our clients like to self-host and someone back in the day made the very smart decision that we should support 3 different JBoss versions (thankfully no more Websphere), and 5 different databases DBs. Something inevitably breaks on some of them. Especially since our test systems don't necessarily use patched versions of JBoss since that would cost money. We could have avoided a lot of stress by going for Spring and 1 or 2 databases instead.
|
|
#
?
Feb 26, 2021 19:12
|
|
|
HamAdams posted:we have a pretty extensive collection of automated qa tests and its always broken. our testers get so excited whenever the pass rate is over like 60%. That upsets me so much. Tests that don't reliably pass are worse than useless because they encourage ignoring failures. Unreliable tests are just as bad as no tests at all.
|
|
#
?
Feb 26, 2021 19:19
|
|
|
At a previous job, we had the double-whammy of fragile Selenium tests combined with our team's QA person being the one who would immediately go in and try to fix the failing test, no matter whose team actually caused it (if it was even a real failure). Meanwhile, our team's "In Review" column in JIRA would be beet red with sirens going off, because it was way over the ticket limit. The number of times we had to gently (or not so gently) prod the QA person to delegate/deflect so we could get anything done, ugh.
|
|
#
?
Feb 26, 2021 19:24
|
|
|
A key to having End to End automated testing particularly as part of a CI process, is someone has to be responsible for non-deterministic test failures from the infrastructure. If the engineers working on the project see constant failures in code they didn't change it undermines the confidence in the testing.
|
|
#
?
Feb 26, 2021 19:33
|
|
|
ChickenWing posted:How do you distinguish e2e tests from automated QA tests? Or is QA automation a pipe dream doomed to be too brittle forever? My entire job is basically trying to answer the question "How do we use software to better improve test". We get an enormous amount of freedom to experiment, and experiment we have. Here's what I've more or less decided on: As a product's state is more and more finite, automated tests can cover more of your total test area. Whether or not you can realize returns on that automation vs human effort in a given time span is dependent on the product and processes involved, but even in the most optimistic of scenarios, automated testing is going be unable cover everything. You can definitely reduce the human effort required to complete QA with confidence, but you can't eliminate it. Even in the best of times, it's hard to build rigid, reliable automated tests that aren't exactly unit tests in code (and even that is hardly the Perfect Solution™). Anyone who insists they can remove all human effort from that process is trying to sell you something... Consider the opposite end of the spectrum of this problem, where possible state approaches infinity. Something like a video game becomes overwhelming to test really quickly without human QA. To this end, the goal we have is not to eliminate people from the process, but instead remove overhead that comes with test. Asking questions like, 'What tooling exposes the most useful data to discover bugs' and 'What is the absolute path of least resistance between bug discovery, entry, and fix verification'? have gone a long way to reducing man hours in test without sacrificing quality. There are pieces of test you can automate, but if you are in game QA Engineering for a game at large scale, and your task is to automate, I'd argue your team is solving the wrong problem. There is a lot of nuance to this and it depends a lot on the nature of the product. I'd argue strongly that most of the solutions a team develops should be product specific (if scale allows). I also think the way most QA Engineering is done is quite bad, especially in games. I should write a proposal for GDC... Canine Blues Arooo fucked around with this message at 22:40 on Feb 26, 2021 |
|
#
?
Feb 26, 2021 22:32
|
|
|
New Yorp New Yorp posted:That upsets me so much. Tests that don't reliably pass are worse than useless because they encourage ignoring failures. Unreliable tests are just as bad as no tests at all. totally agree, its been brought up in almost every retro since i started this job but its not something really under our control. our framework is bad enough that id probably look for a new job if i were a tester.
|
|
#
?
Feb 26, 2021 22:41
|
|
|
CPColin posted:At a previous job, we had the double-whammy of fragile Selenium tests combined with our team's QA person being the one who would immediately go in and try to fix the failing test, no matter whose team actually caused it (if it was even a real failure). Meanwhile, our team's "In Review" column in JIRA would be beet red with sirens going off, because it was way over the ticket limit. I know you mentioned this story before, but for the bolded part, would they go and make the test pass and check it in?
|
|
#
?
Feb 26, 2021 22:51
|
|
|
A lot of the tests had Thread.sleep() everywhere, desperately trying to get some asynchronous action to complete before an assertion came along. In these cases, the QA person on my team tended to "solve" the problem by increasing the sleep delay. In other cases, the problem was because somebody changed the HTML and nobody updated the tests to match (or ran them to realize they needed to). For some reason, instead of instantly bouncing a ticket to the team that caused the problem, our QA person would dig deep into it, trying to figure out what went wrong. The worst, though, were when somebody went, "Oh, whoops, I forgot a database update! Hang on." in chat, but our QA person was already too deep in Thread.sleeps() to notice. A lot of the problem was the weird siloing of responsibility; the designer doing the HTML and CSS and the developer doing the backend functionality had explicit permission to leave the testing to the QA person on their team, so there was no shared ownership of the testing code. When the QA person on our team, who actually was the person tasked with improving the overall reliability of the testing, I held up the meeting where they were announcing their departure by insisting that test code had to become a joint effort that the rest of us couldn't just brush off onto QA any more. I got fired like two weeks later, lol
|
|
#
?
Feb 26, 2021 23:20
|
|
|
Pentagon Wars - Bradley Fighting Vehicle Evolution
|
|
#
?
Feb 27, 2021 19:41
|
|
|
BTW What’s the rest of that movie about? It seems like the 10 minute clip covers the subject pretty well.
|
|
#
?
Feb 27, 2021 19:56
|
|
|
smackfu posted:BTW What’s the rest of that movie about? It seems like the 10 minute clip covers the subject pretty well. Discovering the full extent of the Bradley's flaws and getting it live fire tested and embarrassing the general responsible in front of a congressional hearing. But the general and his cronies still get promoted and the guy trying to uncover the corruption just forced to retire. Can't beat the system from within.
|
|
#
?
Feb 28, 2021 02:51
|
|
|
Canine Blues Arooo posted:My entire job is basically trying to answer the question "How do we use software to better improve test". We get an enormous amount of freedom to experiment, and experiment we have. This is a great philosophy, particularly the bits in bold (even in non-game applications). When I'm asked about test automation, my question is always 'is your software going to be used by machines, or by people?'. An automated test is great for ensuring certain bases are covered, but it's harder for them to determine quality. In my case, I work in a very small company, and I'm not a particularly good developer (as in, I am a manual tester who happens to know enough to cobble together tests from a pattern taken from the internet), and sometimes the problems I ran into I wasn't sure if it was just that I wasn't particularly good at C#/Selenium, or if they were a problem that needed to be handed onto someone else. E.g. The steps worked fine manually, but the automated test was not moving from Step 4 to Step 5 and I lacked the skill or experience to work out if I needed to Thread.sleep() my way out of it, or if a developer could do something to get me past the issue.
|
|
#
?
Feb 28, 2021 04:05
|
|
|
Best case end-to-end tests I’ve worked with was on an old school monolith app that did everything itself except using an external database and mail server. In the E2E tests, it would spin up an empty database, load fresh data into it, and then run completely independent test runs, resetting the database between each. There was also the team expectation that every commit had the appropriate tests updated, and all the tests ran at push time and you had to fix them if you broke them. And it still sucked because the tests were fragile and took twenty minutes to run.
|
|
#
?
Feb 28, 2021 14:25
|
|
|
Writing e2e tests that aren't super brittle and unreliable is very difficult, and it's also difficult to learn the general principles and to gain practice writing them, and I'm not aware of any substantial literature on the subject either. On top of that, the tools tend to be either very verbose or very glib, or somehow both, which will make you want to design and write extensive page objects, which is challenging in its own right and not the sort of thing you want to be busy with while you're learning the thing your bespoke library wraps. They're always going to be slow, and you're always going to have to deal with complicated data setup and cleanup. However, they can give you the most certainty you can attain (from an automated process) about whether all the different components of your application are actually doing what you intended. And at the end of the day, it's only a pretty ordinary kind of difficult.
|
|
#
?
Mar 1, 2021 05:52
|
|
|
Bongo Bill posted:However, they can give you the most certainty you can attain (from an automated process) about whether all the different components of your application are actually doing what you intended. And at the end of the day, it's only a pretty ordinary kind of difficult. E2E will also stress test unhappy paths in the precise ways you don't want: by surfacing transient failures in ways that prevent you from using measures like "build is yellow" as a useful status indicator.
|
|
#
?
Mar 5, 2021 02:20
|
|
|
How do you deal with a bad new job? A couple of months ago, I left a low paying but chill job for a six figure position. The money is great, but everything else is terrible. For instance: 1) I'm stuck in meetings (stand ups and other ceremonies) from 9 to 12. 2) I'm new, so I can't really do anything on my own yet. But when I try to ask questions, they are either too busy or just straight up ignore me. I'm thinking about starting the job hunt again, but should I give it more time?
|
|
#
?
Mar 5, 2021 14:53
|
|
|
Are you stuck in meeting 3 hours every day of the week or Is it only for start and end of spring ceremonies ? 3 hours of meeting everyday seem way overkill for an individual contributor... Is there a sandbox environment where you can fudge around ? Did your boss give you some objectives ?
|
|
#
?
Mar 5, 2021 15:03
|
|
|
I've been put on multiple projects, each with their own standup and ceremonies. This adds up to my morning calendar being booked pretty much everyday. I do have a local environment, but the code is opaque. It's hard for me, as a beginner, to make heads or tails of it on my own. I guess objective is to get my tickets done.
|
|
#
?
Mar 5, 2021 15:23
|
|
|
Ither posted:2) I'm new, so I can't really do anything on my own yet. But when I try to ask questions, they are either too busy or just straight up ignore me. IMO this is a huge red flag and reason enough to  as soon as possible. If people aren't helping you because they don't have the time then management has failed, deliberately or otherwise, to allow for on-boarding and assisting new hires (This may have something to do with the extreme amount of meetings you mentioned). If people aren't helping because it's the culture then yikes.
|
|
#
?
Mar 5, 2021 15:29
|
|
|
Three hours of meetings every day is a management failure.
|
|
#
?
Mar 5, 2021 15:32
|
|
|
|
| # ? Apr 26, 2024 15:25 |
|
|
Ither posted:I'm thinking about starting the job hunt again, but should I give it more time? My vote is to go ahead and start looking. If the current job unfucks itself while you're looking, bonus. I emerged from seven months of unemployment with a job that did similar things to what you're describing. It was just starting a switch to Scrum, so the first week was all-day Scrum training. The instructor basically said, "It doesn't sound like this is compatible with your organization." but management had sunk the cost and pressed onward. Each of us was siloed in our assigned projects with no collaboration, so all the planning meetings were pretty tiresome and useless. I'd ask things of my coworkers like, "What package should this class go in?" and get a meeting invitation for the next morning to talk about it. I started looking again after a month and took my current job three months after that. Current job isn't a ton better, since I'm the only developer, but at least I'm calling the shots now and don't have to sit through sprint planning!
|
|
#
?
Mar 5, 2021 15:47
|
|