Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us $3,400 per month for bandwidth bills alone, and since we don't believe in shoving popup ads to our registered users, we try to make the money back through forum registrations.
«187 »
  • Post
  • Reply
CommieGIR
Aug 22, 2006

If Godzilla can do it, you know I can deliver!

"We don't need QA/UAT, we just need to pound out a really insecure and buggy product! Jackpot!"

Adbot
ADBOT LOVES YOU

fart-powered cars
Apr 19, 2001

I WENT VIRAL AND ALL I GOT WAS THIS LOUSY AVATAR


hobbesmaster posted:

are these all done over cellular? the network itself provides a lot of security in that case at least. of course until someone gets your password to control center/command center

they're done over cell and wifi, depending on how big or urgent the update is. the network does generally provide security in the transport sense - the backend systems are what worry me and tesla is a big enough target that a determined actor could gain control. i can only guess how the likes of NIO (a chinese e-car startup) runs their backend.

fart-powered cars
Apr 19, 2001

I WENT VIRAL AND ALL I GOT WAS THIS LOUSY AVATAR


CommieGIR posted:

"We don't need QA/UAT, we just need to pound out a really insecure and buggy product! Jackpot!"

we had firmware QA and they were great folks. for the lack of staff and timelines we had they were amazing people and the folks i worked directly with, save a few, were really talented. that said, it takes more than just firmware to QA something like a car that talks to a complex backend with continuous delivery into production

mystes
May 31, 2006



Xealous posted:

others didn't pass because they were harness people working in cad programs and not actually programmers
Ok I'm now actually impressed that Tesla has managed to successfully ship any cars that at least work some of the time.

Farmer Crack-Ass
Jan 2, 2001

ensign dwarf, set a course for getting blitzed the fuck out

warp factor nine

Xealous posted:

more fun facts:

the infotainment system and gateway don't have a battery-backed rtc. when the system reboots (sleep, deep sleep, reboot, whatever) the car is at tyool 1970 until it gets ntp again. the logs themselves are written in a binary ring buffer format and when they come in they used to end up in a giant 700TB single mysql database after they were expanded. all of production after-sales service and engineering relies on that single log interpretation system which ran on centos 5 and python 2.4 until hbase/hadoop and friends were brought in.

the supercharger system uses ssh dss keys to "vpn" back to the datacenter to a single server over 2G wireless with very limited resources. the connection is essentially simplex for various reasons so getting data to and from the supercharger is usually a 1KB/s operation unless that site has had connection aggregation done. at one point i looked at the system and to pull data out for analysis, somebody had written a bash script that was printf'ing in a for loop across ~5k devices. it would usually take about 3 days to do a successful firmware update on any single supercharger.

we once patched openssl to ignore client cert expiry because somebody forgot to create a process to update keys in the field and all the customer cars started falling offline because their certs had expired. the quick and dirty was to just patch openssl quickly and make openvpn on the server side use that one while we created those processes for about 2 weeks.

Xealous posted:

yep that was 90% of it. most of the time me and the other firmware folks were chasing elon's whims about what to do with firmware. where i should have been fixing critical issues in the system i was pulled off to do poo poo like add farting unicorns

Xealous posted:

they aren't the first - for what we were doing at the time it made sense and helped us get the program off the ground quickly. lots of room for improvement and in 8 years, they should have done so.

my issue was the fact that the systems doing the flashing were running the yocto images and perl and the guy writing the perl was also responsible for writing the thing that actually updates the car. that thing (the car-side updater) is about ~100k lines of C in a single file. code reviews were always a laugh riot

Xealous posted:

99% of what i'm talking about is "public" anyway. tesla isn't encrypting their firmware and it's really easy to glean information from the vpn with a packet cap because nothing inside the vpn (was) encrypted. dumping tegra 3 model s and x is trivial and tesla's cars are nowhere near as secure as they'd have you believe.

for example, at one time you were able to root a model s with a usb stick and a gstreamer exploit.


these are extremely fun facts, thank you

fart-powered cars
Apr 19, 2001

I WENT VIRAL AND ALL I GOT WAS THIS LOUSY AVATAR


the early days of tesla, post-roadster, early model s and the start of model x were good times - everyone was trying to prove the technology worked, we were innovating and making something that hadn't been done before. things really started to poo poo the bed around the time we pivoted from model 3 plans to shipping model x first. the falcon wing doors were _such a shitshow_. they ended up delaying the program almost a year, hence why model 3 basically skipped all the usual phases a car goes through for validation. i mean, come on - you have bumpers falling off in the rain, the interior is a disaster, there's no instrument cluster which takes your eyes off the road - this list just goes on.

Wulfolme
May 10, 2008



Lipstick Apathy

smothering your rising company to have the car doors of a billionaire

amazing

Sagebrush
Feb 26, 2012




Gravy Boat 2k

I'm suddenly realizing that the model X would have been in its early design phases right around the time that episode aired

Rex-Goliath
Nov 12, 2016

HIS ROYAL MAJESTY


oh wow lotta new posts in the grimes boyfriend thread wonder what’s goi-


Fuzzy Mammal
Aug 15, 2001



Lipstick Apathy

hell of good poo poo, op.

FAUXTON
Jun 2, 2005

daef


CommieGIR posted:

"We don't need QA/UAT, we just need to pound out a really insecure and buggy productship half a finished product to buy time for finishing the other half as updates! Jackpot Agile!"

elite_garbage_man
Apr 3, 2010


ELITE PENS FOR AN ELITE GARBAGE MAN

SKILCRAFT
QUALITY BLIND MADE PRODUCTS

Jesus christ this is loving wild.

Zamujasa
Oct 27, 2010




Bread Liar

mr musk's wild ride

Blue Train
Jun 17, 2012
Probation
Can't post for 3 days!


Xealous posted:

99% of what i'm talking about is "public" anyway. tesla isn't encrypting their firmware and it's really easy to glean information from the vpn with a packet cap because nothing inside the vpn (was) encrypted. dumping tegra 3 model s and x is trivial and tesla's cars are nowhere near as secure as they'd have you believe.

for example, at one time you were able to root a model s with a usb stick and a gstreamer exploit.

jeeztreamer

Zikan
Feb 29, 2004

Metal Gear?


https://twitter.com/elonbachman/sta...8499752963?s=21

akadajet
Sep 14, 2003



Slippery Tilde

Lutha Mahtin posted:

wait, do you mean the guy came in and cleaned out the department of people who couldn't pass his personal fizzbuzz pointer quiz?

lol

Endless Mike
Aug 13, 2003

Get running
Start pumping your bunions, I'm coming
I'm the dumbest, who flamethrow your function to Funyons
Flame your crew quicker than Trump fucks his youngest
Now face the flame fuckers your fame and fate's done with



selling demo cars is very common, and they do, in fact, count as "new" since they've never had an initial say, but dealerships will give steep discounts on them, since they'll usually have a few hundred miles on them. something tells me tesla is not doing that.

Dixie Cretin Seaman
Jan 22, 2008

all hat and one catte


Hot Rope Guy

Cocoa Crispies
Jul 20, 2001

Vehicular Manslaughter!



Pillbug

Wulfolme posted:

smothering your rising company to have the car doors of a billionaire

amazing

Sagebrush posted:

I'm suddenly realizing that the model X would have been in its early design phases right around the time that episode aired

lmao

hobbesmaster
Jan 28, 2008



Endless Mike posted:

selling demo cars is very common, and they do, in fact, count as "new" since they've never had an initial say, but dealerships will give steep discounts on them, since they'll usually have a few hundred miles on them. something tells me tesla is not doing that.

this one wasn't new though; theres actually a pretty big legal difference between a new car and a used car. apparently this one had been titled and registered so its no longer new

you can buy a "new" car with 200 miles on it, this wasn't a "new" car by law though

Endless Mike
Aug 13, 2003

Get running
Start pumping your bunions, I'm coming
I'm the dumbest, who flamethrow your function to Funyons
Flame your crew quicker than Trump fucks his youngest
Now face the flame fuckers your fame and fate's done with


hobbesmaster posted:

this one wasn't new though; theres actually a pretty big legal difference between a new car and a used car. apparently this one had been titled and registered so its no longer new

you can buy a "new" car with 200 miles on it, this wasn't a "new" car by law though

yeah, definitely. that tweet doesn't make that entirely clear.

Suspicious Dish
Sep 24, 2011



Fun Shoe


"Evolutionary theory says that diverse species never collaborate. People only want to take care of their families."

do you think this guy even has a family lol

akadajet
Sep 14, 2003



Slippery Tilde

this thread got really good when this xealous lurker showed up

President Beep
Apr 29, 2009


Grahms ah dint fell so gud!


akadajet posted:

this thread got really good since this xealous lurker showed up

yeah, their posts really add some gravitas.

akadajet
Sep 14, 2003



Slippery Tilde

Xealous posted:

appreciate the offer, I mostly browse and don’t post because I’m a boring computer toucher

may I offer some advice?

fart-powered cars
Apr 19, 2001

I WENT VIRAL AND ALL I GOT WAS THIS LOUSY AVATAR


tesla basically runs their entire business like a just in time compiler only they don't treat warnings or errors as failures. most groups in the company don't cross-communicate so there's a lot of duplication of effort.

i once got pulled into a meeting because a car burned down when it was attached to a supercharger and we didn't get a log out of the car. normally under some emergency circumstances the car will try to upload a log when it thinks poo poo has gone really badly, but in this particular case it was far enough away from a tower it had half 3G connection and had to upload a 30MB log via HTTPS POST. the car burned down before it even got to 10MB and the system was only designed for exponential backoff retries, not resumption of in-progress. elon was calm about it, but we had to justify why we never had time to address it - maybe it was because we were all busy making unsafe features work?

fart-powered cars
Apr 19, 2001

I WENT VIRAL AND ALL I GOT WAS THIS LOUSY AVATAR


also on the supercharger note - you can get blacklisted from using them if you charge on them all the time. that's because the supercharger bypasses the charging regulator boards and dumps directly into the pack at 300A/450v which creates a ton of wear on the battery. want to keep your range high? don't supercharge often.

President Beep
Apr 29, 2009


Grahms ah dint fell so gud!


do they define “too often”?

hobbesmaster
Jan 28, 2008



Xealous posted:

tesla basically runs their entire business like a just in time compiler only they don't treat warnings or errors as failures. most groups in the company don't cross-communicate so there's a lot of duplication of effort.

i once got pulled into a meeting because a car burned down when it was attached to a supercharger and we didn't get a log out of the car. normally under some emergency circumstances the car will try to upload a log when it thinks poo poo has gone really badly, but in this particular case it was far enough away from a tower it had half 3G connection and had to upload a 30MB log via HTTPS POST. the car burned down before it even got to 10MB and the system was only designed for exponential backoff retries, not resumption of in-progress. elon was calm about it, but we had to justify why we never had time to address it - maybe it was because we were all busy making unsafe features work?

good news! 3G is dead too!

GWBBQ
Jan 2, 2005



what is elon like when stuff goes wrong due to his idiotic micromanagement and big stupid ideas?

fart-powered cars
Apr 19, 2001

I WENT VIRAL AND ALL I GOT WAS THIS LOUSY AVATAR


President Beep posted:

do they define “too often”?

algorithm-based now - the ai poo poo i was working on took into account a lot of factors to determine if you were abusing it before i left. the criteria takes into account the state of many components in the car, your driving patterns and other details. or it did anyway. not even sure that stuff is running still - they rotated projects in and out of existence pretty rapidly.

fart-powered cars
Apr 19, 2001

I WENT VIRAL AND ALL I GOT WAS THIS LOUSY AVATAR


GWBBQ posted:

what is elon like when stuff goes wrong due to his idiotic micromanagement and big stupid ideas?

he's never wrong. his "open door policy" was an invitation to catch you breaking rank.

fart-powered cars
Apr 19, 2001

I WENT VIRAL AND ALL I GOT WAS THIS LOUSY AVATAR


tesla was also in the news because they were doing cute poo poo like spinning up k8s clusters which had AWS IAM access to sensitive S3 buckets but wasn't ssl'd and the k8s mgmt api was available publicly. there were other teams running industrial control equipment with centos 7 an no hardening at all.

there was one time where a canadian kid stole the domain and redirected emails and managed to take over slack and a bunch of other poo poo because the idiot IT team didn't hide the registrar information or use something like markmonitor. the car-side stuff at least did full mtls at the time so it was ok, but lol did that kid get a lot of info.

Bulgakov
Mar 8, 2009


lol this is all so tasty, thanks

Wulfolme
May 10, 2008



Lipstick Apathy

this is going to get poached for one of the $TSLAQ twitters within days

maybe

C.H.O.M.E.
Dec 8, 2000

MVP
Most Valuable Poster Awarded 2002, 2005, 2007


Cybernetic Crumb

Xealous posted:

algorithm-based now - the ai poo poo i was working on took into account a lot of factors to determine if you were abusing it before i left. the criteria takes into account the state of many components in the car, your driving patterns and other details. or it did anyway. not even sure that stuff is running still - they rotated projects in and out of existence pretty rapidly.

thats just what i want, the car manufacturer monitoring how i drive the car i own and deciding that features should be turned off after i have purchased it, that's a good feature.

fart-powered cars
Apr 19, 2001

I WENT VIRAL AND ALL I GOT WAS THIS LOUSY AVATAR


C.H.O.M.E. posted:

thats just what i want, the car manufacturer monitoring how i drive the car i own and deciding that features should be turned off after i have purchased it, that's a good feature.

you have no idea. any connected car is ripe for data harvesting and you (the consumer) should expect it going forward. on that note, china has a law in place that mandates all electric cars send real time telemetry to their government servers - model s/x/3, NIO cars and any other electric car if they're driving already complies with that law to be road certified. don't be surprised if that becomes a mandate in other countries

Roosevelt
Jul 18, 2009

I'm looking for the man who shot my paw.


this thread owns

C.H.O.M.E.
Dec 8, 2000

MVP
Most Valuable Poster Awarded 2002, 2005, 2007


Cybernetic Crumb

Xealous posted:

you have no idea. any connected car is ripe for data harvesting and you (the consumer) should expect it going forward. on that note, china has a law in place that mandates all electric cars send real time telemetry to their government servers - model s/x/3, NIO cars and any other electric car if they're driving already complies with that law to be road certified. don't be surprised if that becomes a mandate in other countries

thats ok my car is 15 years old and i will never buy a new one and the only thing i connect it to is my own butt

Adbot
ADBOT LOVES YOU

creationist believer
Feb 16, 2007



College Slice

C.H.O.M.E. posted:

thats just what i want, the car manufacturer monitoring how i drive the car i own and deciding that features should be turned off after i have purchased it, that's a good feature.

there are four essential freedoms that the drivers of cars should always have

freedom 0 is the freedom to drive the car however you wish

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply
«187 »