Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Locked thread
A Wheezy Steampunk
Jul 16, 2006

High School Grads Eligible!
big data, small data, this thread is about accepting data of all sizes

maybe talk about your favorite data formats and tools like csv and pandas

Adbot
ADBOT LOVES YOU

Smythe
Oct 12, 2003
sand king my favorite data hero

triple sulk
Sep 17, 2014



its real big op

power botton
Nov 2, 2011

its not the size of the data its how you use it!!!!

pram
Jun 10, 2001
data from star trek . lol. tng. haha.



lmfao

power botton
Nov 2, 2011

one thing i hope to get fromi this thread is an exact defition of big data. is it 1 gig? 100 gigs? 1000 gigs!?!?! im sure theres people with more gigs than that but theyre pr9obably pretty few and far between.

when does data stop being data... and starts being real... ly big. really big data.

Smythe
Oct 12, 2003
data is googe

pram
Jun 10, 2001
mappe reduce all dattas - goog

A Wheezy Steampunk
Jul 16, 2006

High School Grads Eligible!

power botton posted:

one thing i hope to get fromi this thread is an exact defition of big data. is it 1 gig? 100 gigs? 1000 gigs!?!?! im sure theres people with more gigs than that but theyre pr9obably pretty few and far between.

when does data stop being data... and starts being real... ly big. really big data.

i think big data is is more than 5TB but that's just by my country reckoning

Captain Foo
May 11, 2004

we vibin'
we slidin'
we breathin'
we dyin'

A Wheezy Steampunk posted:

i think big data is is more than 5TB but that's just by my country reckoning

but what kind of data, my prod SAN has 21TB in use and it's not remotely big data

but if that was a single DB then yeah maybe we talkin big data

power botton
Nov 2, 2011

this brings up a good point. does big data also have a time component>? likew growing to 5000 gigs over a while isnt bad but what if you get 5000 gigs in a week?? does big data have to be structured or is a hard drive big data?!

A Wheezy Steampunk
Jul 16, 2006

High School Grads Eligible!

Captain Foo posted:

but what kind of data, my prod SAN has 21TB in use and it's not remotely big data

but if that was a single DB then yeah maybe we talkin big data

yeah i'm thinking 5TB of actual information not a database with 5 1TB video files

Captain Foo
May 11, 2004

we vibin'
we slidin'
we breathin'
we dyin'

power botton posted:

this brings up a good point. does big data also have a time component>? likew growing to 5000 gigs over a while isnt bad but what if you get 5000 gigs in a week?? does big data have to be structured or is a hard drive big data?!

hard drives are p small these days, forums poster power botton

Captain Foo
May 11, 2004

we vibin'
we slidin'
we breathin'
we dyin'

A Wheezy Steampunk posted:

yeah i'm thinking 5TB of actual information not a database with 5 1TB video files

why would you even put the latter in a DB

power botton
Nov 2, 2011

Captain Foo posted:

hard drives are p small these days, forums poster power botton

this is a common misconception, actually hard drives have never ben bigger. its never been easier to store all your data on hard drives.

Mr. Apollo
Nov 8, 2000

day-ta or da-ta?

power botton
Nov 2, 2011

https://www.youtube.com/watch?v=q_noTEIsaPk

Captain Foo
May 11, 2004

we vibin'
we slidin'
we breathin'
we dyin'

power botton posted:

this is a common misconception, actually hard drives have never ben bigger. its never been easier to store all your data on hard drives.

physically smaller my man

OldAlias
Nov 2, 2013

pram posted:

data from star trek . lol. tng. haha.



lmfao

https://www.youtube.com/watch?v=BomG2xGhxVY

:xd:

graph
Nov 22, 2006

aaag peanuts
its loving hard planning for data when no one across a department can give an estimate within orders of magnitude of what they might need

Captain Foo
May 11, 2004

we vibin'
we slidin'
we breathin'
we dyin'

graph posted:

its loving hard planning for data when no one across a department can give an estimate within orders of magnitude of what they might need

not enough jiggabites

carry on then
Jul 10, 2010

by VideoGames

(and can't post for 10 years!)

here's some data for the thread: 6, 4, 3

Captain Foo
May 11, 2004

we vibin'
we slidin'
we breathin'
we dyin'

carry on then posted:

here's some data for the thread: 6, 4, 3

that's some good data

graph
Nov 22, 2006

aaag peanuts

Captain Foo posted:

not enough jiggabites

like the persons assigned to data collection dont understand the concept of file size. its weird

pram
Jun 10, 2001

carry on then posted:

here's some data for the thread: 6, 4, 3

thanks i will input this into our mongodb instance asap

Captain Foo
May 11, 2004

we vibin'
we slidin'
we breathin'
we dyin'

pram posted:

thanks i will input this into our mongodb instance asap

welp data's gone, thanks pram

Captain Foo
May 11, 2004

we vibin'
we slidin'
we breathin'
we dyin'

graph posted:

like the persons assigned to data collection dont understand the concept of file size. its weird

lol

graph
Nov 22, 2006

aaag peanuts

dont get me started on data lifecycle management either

i can tell you EXACTLY how stuxnet got on that poo poo

Captain Foo
May 11, 2004

we vibin'
we slidin'
we breathin'
we dyin'

graph posted:

dont get me started on data lifecycle management either

i can tell you EXACTLY how stuxnet got on that poo poo

imma plug this usb directly into my san lmfao

suffix
Jul 27, 2013

Wheeee!

pram posted:

data from star trek . lol. tng. haha.



lmfao

rip lennart nimoy

Captain Foo
May 11, 2004

we vibin'
we slidin'
we breathin'
we dyin'

suffix posted:

rip lennart nimoy

another casualty of systemd

Squinty Applebottom
Jan 1, 2013

whats the difference between a san and nas

Squinty Applebottom
Jan 1, 2013

one is a saint and the other is a rapper?

graph
Nov 22, 2006

aaag peanuts

Captain Foo posted:

imma plug this usb directly into my san lmfao

:sigh:

Captain Foo
May 11, 2004

we vibin'
we slidin'
we breathin'
we dyin'

Squinty Applebottom posted:

whats the difference between a san and nas

greyforums are thatway ->

graph
Nov 22, 2006

aaag peanuts
crossposting

Captain Foo
May 11, 2004

we vibin'
we slidin'
we breathin'
we dyin'

graph posted:

crossposting

that's a good poster

dragon enthusiast
Jan 1, 2010

graph posted:

its loving hard planning for data when no one across a department can give an estimate within orders of magnitude of what they might need

doesn't matter they'll still fill it all up with garbage within 90 days

shyduck
Oct 3, 2003


why can't the enemy be a she :colbert:

Adbot
ADBOT LOVES YOU

The Management
Jan 2, 2010

sup, bitch?

Squinty Applebottom posted:

whats the difference between a san and nas

nas is a backwards san. makes u think

  • Locked thread