Go: Google's Plan9 refugees are staging a breakout

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Go: Google's Plan9 refugees are staging a breakout

«‹›14 »

homercles: Feb 14, 2010

Ignoring the "compile struct to parsing code" alternatives, some dynamic stuff:

1. https://github.com/jmoiron/jsonq
2. https://github.com/valyala/fastjson
3. https://github.com/stretchr/objx
4. https://github.com/yalp/jsonpath

Ranked in order of interestingness.

# ? Jul 9, 2020 12:06

Adbot: ADBOT LOVES YOU

# ? Apr 18, 2024 11:15

Blame Pyrrhus: May 6, 2003; Me reaping: Well this fucking sucks. What the fuck.; Pillbug

geonetix posted:

There's typicall two schools of thought about the json stuff in go as far as I've seen: declare your expected responses explicitely and unmarshal into that, or use the map method.

I'm a strong believer of the first method with predefined expected structs/types. Not only do you need to know what you're getting anyway, so you can actually do something meaningful with it, but it also helps you find changes, malformed input, reject nonsense, etc etc etc. All the code that follows from the json unmarshalling ends up a lot easier afterwards.

Using map[string]interface{} (note, not all json starts with an object, it can be an array, or just a string, or many other valid json types, too. so that would break immediately as well), leads you down a road straight into reflection hell, and personally I think you should avoid that at all cost, or at least isolate it in some tailored libraries so it gives you... structured data that you can predictably work on.

In neither python nor PowerShell have I ever had to pre-define a response structure and then populate the data. I've had to declare classes, which expect a structure and use the resulting REST response object to help populate a class properties. But how do people deal with REST APIs using Go if they have to pre-write structs to deal with the dozens, sometimes hundreds of different responses? By literally writing dozens, sometimes hundreds of structs? Really?

I want to write a helper function to invoke a rest call and leave parsing of the response to the functions that invoke that helper function. JSON in -> object out.

For instance, in PowerShell, I'll write a bespoke Invoke-ProductRestCall function that does all of the heavy lifting (authentication, URI construction and validation, response error handling, etc)

Then I'll write a cmdlet as a function for like, Get-ProductObject that will use the Invoke-ProductRestCall to populate object data.

For a very basic example in PowerShell, look at lines 32 and 33 here. Line 32 invokes a custom rest call to the API and populates a (generic PSObject) $response, and then the cmdlet function returns that response as-is.

At no point do I ever need to pre-define a struct. I get that powershell is often regarded as a weaker language, and I'm obviously no developer so idk, I'm sure it is. But I can't believe that a modern language like Go is this much more cumbersome for basic JSON management.

Blame Pyrrhus fucked around with this message at 17:05 on Jul 9, 2020

# ? Jul 9, 2020 16:07

gariig: Dec 31, 2004; Beaten into submission by my fiance; Pillbug

Linux Nazi posted:

At no point do I ever need to pre-define a struct. I get that powershell is often regarded as a weaker language, and I'm obviously no developer so idk, I'm sure it is. But I can't believe that a modern language like Go is this much more cumbersome for basic JSON management.

Go is very opinionated language. One of those opinions is that it's strongly typed and statically checked, like 99%. Like geonetix said you can use map[string]interface{} but Go is going to fight you every step of the way. Forcing you to cast and check when assigning to something not an interface{}. Go is going to push you to define a struct and say how to marshal the JSON to that struct. You can choose not to do this but since you are going against Go's opinion it's going to make it VERY difficult.

The one up side to Go's opinion is that an "object.foo" was meant to be there because there's a struct that says "foo string" exists and a compiler checked it. With Powershell when you do "object.foo" does it work? Beats me, run the script and see. You can't know that "object.foo" is correct because the check is dynamic and has to checked at run time.

How my job gets around making lots of structs is using Protocol Buffers and letting the protobuf library handle the marshal/unmarshal of data (both JSON and protobuf). I'm sure there's a Swagger->Go struct tool out there which will help. Ultimately, Go has very little support for dynamic programming unlike Python and Powershell which is strongly typed but dynamically checked)

Honestly, pulling JSON into Go is probably the weaker side of the language. It can do it but it's not a great experience. It's better at receiving and returning JSON as a server, which is where the static nature of the JSON marshalling is great. Much easier to see where things changed IMO.

As far as Powershell is a "weaker language" it's a language that is trying to solve a different problem. Doesn't make it better or worse and it's probably the "better" language for talking to random JSON APIs.

# ? Jul 9, 2020 18:25

Blame Pyrrhus: May 6, 2003; Me reaping: Well this fucking sucks. What the fuck.; Pillbug

gariig posted:

Go is very opinionated language. One of those opinions is that it's strongly typed and statically checked, like 99%. Like geonetix said you can use map[string]interface{} but Go is going to fight you every step of the way. Forcing you to cast and check when assigning to something not an interface{}. Go is going to push you to define a struct and say how to marshal the JSON to that struct. You can choose not to do this but since you are going against Go's opinion it's going to make it VERY difficult.

The one up side to Go's opinion is that an "object.foo" was meant to be there because there's a struct that says "foo string" exists and a compiler checked it. With Powershell when you do "object.foo" does it work? Beats me, run the script and see. You can't know that "object.foo" is correct because the check is dynamic and has to checked at run time.

How my job gets around making lots of structs is using Protocol Buffers and letting the protobuf library handle the marshal/unmarshal of data (both JSON and protobuf). I'm sure there's a Swagger->Go struct tool out there which will help. Ultimately, Go has very little support for dynamic programming unlike Python and Powershell which is strongly typed but dynamically checked)

Honestly, pulling JSON into Go is probably the weaker side of the language. It can do it but it's not a great experience. It's better at receiving and returning JSON as a server, which is where the static nature of the JSON marshalling is great. Much easier to see where things changed IMO.

As far as Powershell is a "weaker language" it's a language that is trying to solve a different problem. Doesn't make it better or worse and it's probably the "better" language for talking to random JSON APIs.

This makes some sense, and honestly when I re-read my last post it comes across as more angry than I intended. It's an approach that - as an operations person who writes code in service of automating "things" like (IaaS deployments) - seems counter-intuitive, but that's mostly because I am just dumb.

Ultimately my goal with learning Go was to author a basic terraform provider for an API-driven storage platform, and at a high level everything about how a terraform provider is written is pretty straight forward. provider() and main() functions largely seem to serve as schemas that map the resource -> resourceFunction relationship, and downstream from that you write the CRUD functions to do all of the heavy lifting. This is going to involve modeling a lot of REST calls to our platform, so I've been looking more closely at the parts involved with doing that specifically.

You're right about Go fighting you though, it's pretty easy to create an inferred "struct" of sorts by declaring a map[string]interface{} but the moment I try to do something like, access a pointer by name within that map, or range loop through over an 'interface{}' type, lint just yells at me and calls me an idiot. Which apparently I am.

As for the $object.foo in PowerShell, not sure if this is what you mean, but you can enforce foo as string when you create the class. It's flexible, either let the object infer it's own type for foo via convert when it doesn't matter, or if it does matter just build a class and enforce the type like:

code:

class testClass {
    [string] $foo
}
$object = [testClass]::new()
$object.foo = "string"

Trying to set $object.foo to an invalid value will get you red ink. One thing I really have grown to like about PowerShell is how flexable the class declarations are as well as being able to natively and super-easily enforce classes and types for parameters.

Blame Pyrrhus fucked around with this message at 22:14 on Jul 9, 2020

# ? Jul 9, 2020 22:08

Vanadium: Jan 8, 2005

Yeah, if you use interface{} you have to cast to some other type (probably a nested map[string]interface{}) all the time. Writing down struct definitions is annoying if you have many/complicated json responses and we also mostly deal with that by having both service and caller share autogenerated protobuf structs. But I'd definitely prefer writing a struct definition over a bunch of casting, particularly because it also documents the overall shape of the expected response locally.

# ? Jul 9, 2020 23:08

Xik: Mar 10, 2011; Dinosaur Gum

Linux Nazi posted:

pre-write structs to deal with the dozens, sometimes hundreds of different responses? By literally writing dozens, sometimes hundreds of structs? Really?

Yeah this is what is done in statically typed languages in my experience. Tooling usually makes it easier, like in C#/VS you can just import a huge json structure and it'll create all the classes for you. A couple of refactoring commands later they are all in seperate files with correct formatting and naming conventions applied.

This probably comes down to personal preference but I would for sure prefer a stack of structs over a map of interface{}'s.

# ? Jul 10, 2020 06:25

Literally Elvis: Oct 21, 2013

I use JSON-to-Go for these sorts of tasks.

# ? Jul 10, 2020 17:58

canis minor: May 4, 2011

When I was learning Go, I've written a utility that would parse the json (what is json but a string in a specific format) into separate files with defined structs. Every object was a file, and then properties converted to estabilished types; if there was a nested object - another file. While I could import those files after the program finished running I couldn't import them dynamically - I didn't simply want to generate JSON structs that would be usable next time the program would run - I wanted them now. Then I started playing with plugin - figuring, hey, let's import those structs as plugin, found out plugins don't allow that and you still end up with interface{} because something something and then got fed up with it all. Was a couple of years ago, so if I'm getting something wrong, I blame it on my broke brain.

I guess, my problem with Go was that even with knowledge of different programming languages, looking at the documentation and after going through tutorials and book about programming in Go I still didn't know what was possible and what wasn't, effectively fighting with the language, because yes, same as you I found it weird that modern language can't import a string in a defined format into a usable struct.

canis minor fucked around with this message at 00:58 on Aug 1, 2020

# ? Aug 1, 2020 00:56

Judge Schnoopy: Nov 2, 2005; dont even TRY it, pal

But if you imported a random json into a dynamic type, it wouldn't be typesafe in other functions and you're playing defense with execution-time debugging. That's the whole point of strong typing.

I do a lot of json with Go (rest integrations) and don't find it to be annoying. I'm defining the types I'm using anyway, it's not much extra to tack on a json tag. I also use couchDB as a back end instead of doing whatever silly file storage thing that is above.

# ? Aug 1, 2020 19:07

Blame Pyrrhus: May 6, 2003; Me reaping: Well this fucking sucks. What the fuck.; Pillbug

Yeah I'm just maintaining a types.go in the project and appending it as-needed.

Worked close with an old friend to build out the bones of a SDK and learned a few good practices around dealing with API calls, as well as using mapstructure for dealing with json responses a little more easily. Now I'm having a blast writing functions in a small Go SDK for our platform and using those when I build out resource functions in our terraform provider.

Learning a lot, and can really appreciate how well documented a lot of things in Go (and terraform) are, since I'm not a developer at all but can make sense of what I am reading. Now when I see lint I immediately know where I hosed up.

# ? Aug 20, 2020 01:07

Kuule hain nussivan: Nov 27, 2008

Hello Go-thread. I'm planning my first Go-project with a couple of people and am having a hard time figuring out whether my design choices are sane or not, please help if you have the time.

The plan is to make a backend consisting of several microservices and run it off Google Cloud Run. A Kubernetes Engine would probably be a better choice, but it doesn't have a free tier and we don't want to spend money, so we decided that Cloud Run should be fine for a proof-of-concept / minimum viable product. My background is primarily in C#, so I'm used to singletons and the like. So my plan was to have my main.go file instantiate a single copy of my other classes (logger, db, etc.) and then pass pointers around. For example, my db class does some logging, so after creating an instance of the logger in main, I would then create an instance of the db class and pass a pointer of the logger to the db class, instead of creating a new instance specifically for the db class. To me, this also feels like it would mesh well with Cloud Run, because there's generally no need to create multiple instances of service classes.

Does this seem sensible? It's technically not a strict singleton, since there's nothing limiting me from creating 10,000 loggers in main. Because of this, I think trying to keep a singleton pattern going on might be a bad idea, and if someone else keeps working on the project it'll eventually lead to a situation where you have a mixed bag of a single instance being passed around as a pointer and some classes creating their own instances. Is there a way of forcing a strict singleton pattern with go? Am I overthinking this too much and should I just go gently caress it and implement what feels good to me?

# ? Sep 20, 2020 16:19

spiritual bypass: Feb 19, 2008; Grimey Drawer

What advantage does singleton provide over passing in the logger and DB connections (or storage providers) as args? In my opinion, it's always better for a web server to instantiate things at startup time instead of attempting to do it in a handler. This way you can write tests and benchmarks with full confidence that all of the needed pieces go in as args instead of having to remember to set up other state beforehand.

# ? Sep 20, 2020 16:51

Kuule hain nussivan: Nov 27, 2008

rt4 posted:

What advantage does singleton provide over passing in the logger and DB connections (or storage providers) as args? In my opinion, it's always better for a web server to instantiate things at startup time instead of attempting to do it in a handler. This way you can write tests and benchmarks with full confidence that all of the needed pieces go in as args instead of having to remember to set up other state beforehand.

Yeah, I'm probably misusing the term singleton here, since I don't actually have a way of enforcing that only one instance of the class is created. Rather, the intention of the program is that only a single instance is created upon startup and passed around. I guess this is fine, and my worry that this will lead to mixed use where one class creates a new instance and another uses an instance passed to it is overthinking things.

# ? Sep 20, 2020 17:19

Rectovagitron: Mar 13, 2007; Grimey Drawer

Kuule hain nussivan posted:

Hello Go-thread. I'm planning my first Go-project with a couple of people and am having a hard time figuring out whether my design choices are sane or not, please help if you have the time.

The plan is to make a backend consisting of several microservices and run it off Google Cloud Run. A Kubernetes Engine would probably be a better choice, but it doesn't have a free tier and we don't want to spend money, so we decided that Cloud Run should be fine for a proof-of-concept / minimum viable product. My background is primarily in C#, so I'm used to singletons and the like. So my plan was to have my main.go file instantiate a single copy of my other classes (logger, db, etc.) and then pass pointers around. For example, my db class does some logging, so after creating an instance of the logger in main, I would then create an instance of the db class and pass a pointer of the logger to the db class, instead of creating a new instance specifically for the db class. To me, this also feels like it would mesh well with Cloud Run, because there's generally no need to create multiple instances of service classes.

Does this seem sensible? It's technically not a strict singleton, since there's nothing limiting me from creating 10,000 loggers in main. Because of this, I think trying to keep a singleton pattern going on might be a bad idea, and if someone else keeps working on the project it'll eventually lead to a situation where you have a mixed bag of a single instance being passed around as a pointer and some classes creating their own instances. Is there a way of forcing a strict singleton pattern with go? Am I overthinking this too much and should I just go gently caress it and implement what feels good to me?

Cloud Run's free tier has been great for me. I did move a service to GKE that had some larger memory requirements, but Cloud Run seems to work for everything else.

For loggers, I honestly use package level variables, as they tend to be safe for concurrent use, and don't impact my overall application design very much. For other things, like DB connections, or other service connections, I typically instantiate a Server struct, and provide those connections or configuration to it in the main function. This helps you structure your handlers in a way where it's easier to plug in a fake database for testing, or a fake/in-memory database or service for local development. I find it helps me keep a separation of concerns between different objects in my app.

In general, instantiating things in Go is extremely cheap, so I take advantage of that unless I want persistent connections, which is frequently the case for databases and other services.

If you're new to Go, one of the first things that helped me a lot when learning how to work with the language was that in general, functions should return pointers to structs, and accept interfaces as arguments. Interface definitions should go by where they're consumed, not where they're produced. This will help you keep a good contract at the call-site, and make it easier to change implementations without breaking consumers. As a bonus, it makes things really easy to fake out for unit testing, helping you avoid integration tests except where necessary.

Finally, always run tests with the race detector on, and use go vet. Have fun.

# ? Sep 20, 2020 19:31

FnF: Apr 10, 2008

Does this thread have any recommendations for Go books?

I'm looking for a book that ideally :

covers all of the basics & intermediate stuff about the language & syntax
covers all of the basics & intermediate stuff about how to structure, write & maintain a not-small Go application
contains a thorough reference to most/all of the standard library
is up-to-date (v1.15)

I'm coming from a Java & Ruby background and am trying to learn Go, and some parts are just confusing the hell out of me (modules & local packages, whatever "vendor" and "vgo" are, some bits of syntax, etc.). Is there a good book that covers all of this, or is Go too new/changing too quickly for this?

If it helps, the "Programming Ruby - The Pragmatic Programmers' Guide" is a good example of the sort of book I'm after.

# ? Dec 29, 2020 16:23

Iverron: May 13, 2012

For the basics / syntax either of the following options are good, iirc I made my way through the tour site referencing sections of the book when I wanted to dive deeper into something:

The Go Programming Language - Alan Donovan (https://www.amazon.com/dp/0134190440)
https://tour.golang.org/welcome/1

My current read as I�m attempting to shift from .NET to go:

Concurrency in Go - Katherine Cox-Buday (https://www.amazon.com/dp/1491941197)

# ? Dec 29, 2020 19:50

Pham Nuwen: Oct 30, 2010

The modules system is the hardest-to-understand part of Go imo. It's strict, it's picky, and it can be unintuitive. It's also better than the two previous alternatives (dep, or "just grab latest off github and run with it") so I put up with it, but on our large corporate project we have to fight it from time to time.

On a small project, it should be pretty reasonable to deal with, just follow the wiki.

# ? Dec 29, 2020 20:11

FnF: Apr 10, 2008

Iverron posted:

The Go Programming Language - Alan Donovan (https://www.amazon.com/dp/0134190440)

It says this was published in 2015. Is this still "current"? I've been getting the impression that Go is changing & developing as a language quite quickly, and I don't want to accidentally learn out-of-date syntax & conventions as that can lead to me internalising bad practices as "the way you do things in Go". Which would be bad!

# ? Dec 30, 2020 11:15

Pham Nuwen: Oct 30, 2010

FnF posted:

It says this was published in 2015. Is this still "current"? I've been getting the impression that Go is changing & developing as a language quite quickly, and I don't want to accidentally learn out-of-date syntax & conventions as that can lead to me internalising bad practices as "the way you do things in Go". Which would be bad!

2015 was well into Go 1.x territory, which means nothing you learn there should be invalidated by later updates in the 1.x line. Syntax will not change, but syntax is minor anyway. The standard library calls should all work, although the best practices may have shifted on a few things.

Kernighan is the co-author on that book, and I'll always recommend his work. I haven't read it, but I'm sure it will help you get your head around the concept of goroutines and channels, which seems to be the biggest challenge for a lot of new Go programmers. Then, use the online documentation (https://github.com/golang/go/wiki/Modules helps) to learn how to use the modules system, and you'll be in good stead.

# ? Dec 30, 2020 17:29

FnF: Apr 10, 2008

Alright then. Thank you both for your help!

# ? Dec 31, 2020 11:22

lifg: Dec 4, 2000; <this tag left blank>; Muldoon

I'm new to go, and having trouble with math/big. I'm creating lots of equations, but math/big uses all methods, no functions, so my code is looking hellish.

Like, I want to do this:

code:

(y**2 - x**3 - 7) % p

But I need big/Int. I'd like to do be able to use it like this:

code:

big.Mod(big.Sub( big.Sub( big.Exp(y,2,nil), big.Exp(x,3,nil)), 7), p)

But since big.Int uses methods, I'm creating code that looks like this:

code:

a := new(big.Int).Set(y)
b := new(big.Int).Set(x)
a.Exp(a, big.NewInt(2), nil)
b.Exp(b, big.NewInt(3), nil)
a.Sub(a, b)
a.Sub(a, big.NewInt(7))
a.Mod(a, p)

And that looks ugly to me.

Am I doing this right?

# ? Aug 12, 2021 19:50

Pham Nuwen: Oct 30, 2010

Taking a look at the docs (https://pkg.go.dev/math/big), it seems like those methods all return the result in addition to setting the receiver. So you should be able to use it as you wish; please forgive my slight re-writing:

code:

	xInput := int64(1)
	yInput := int64(2)
	pInput := int64(3)
	x := big.NewInt(xInput)
	y := big.NewInt(yInput)
	p := big.NewInt(pInput)
	z := big.NewInt(0) // use this as the receiver in everything
	out := z.Mod(z.Sub(z.Sub(z.Exp(y, big.NewInt(2), nil), z.Exp(x, big.NewInt(3), nil)), big.NewInt(7)), p)

Personally, I find your example easier to follow; it feels a lot like working on an RPN calculator.

# ? Aug 12, 2021 20:07

lifg: Dec 4, 2000; <this tag left blank>; Muldoon

Thanks. I�ll play around with using a receiver like that, but maybe you�re right and I�ll stick with what I have.

# ? Aug 14, 2021 21:28

Fluue: Jan 2, 2008

Trying to understand an idiomatic approach to a set-once lazy value for a struct.

I have a simple struct:

code:

type MyObject struct {
	ID              uint32 `db:"id"`
	Name            string `db:"name"`
	shiny           *bool
	types           []ObjectType
}

//
func (o *MyObject) IsShiny() bool {
	// TODO: implement randomness
	return o.shiny
}

I'm initializing and hydrating instances of MyObject from data I retrieve in a database and there are often multiple instances that I construct and return to the caller. The shiny field is determined by a random dice roll and is only set once for the lifetime of the struct.

Should I:
1. Pass in an instance of a seeded *rand.Rand() to IsShiny and only calculate it if o.shiny == nil
2. Create a constructor that accepts *rand.Rand() and do the dice roll during construction

My goal is to be able to test the IsSiny function and make sure the `shiny` field can't be changed after it's set.

edit: also, the dice roll does not need to be perfectly random

Fluue fucked around with this message at 03:56 on Sep 5, 2021

# ? Sep 5, 2021 03:51

gariig: Dec 31, 2004; Beaten into submission by my fiance; Pillbug

Fluue posted:

Should I:
1. Pass in an instance of a seeded *rand.Rand() to IsShiny and only calculate it if o.shiny == nil
2. Create a constructor that accepts *rand.Rand() and do the dice roll during construction

My goal is to be able to test the IsSiny function and make sure the `shiny` field can't be changed after it's set.

edit: also, the dice roll does not need to be perfectly random

If you got with option 1 every time someone calls IsShiny a *rand.Rand() has to be passed in even if shiny has been set.

I would go with option 2 and I would have two constructors. One is internal that�s used by the tests where a rand.Rand can be passed in. The exported constructor would not take a rand.Rand, make one, and use the internal constructor. This keeps it so your test (if in the same package) can set a predetermined rand.Rand but the caller doesn�t get to set anything.

There�s no way to guarantee shiny isn�t changed in some code path internally. Go doesn�t have the concept of const or readonly. So basically you just have to make sure nothing sets it outside the constructor

# ? Sep 5, 2021 04:25

cruft: Oct 25, 2007

gariig posted:

If you got with option 1 every time someone calls IsShiny a *rand.Rand() has to be passed in even if shiny has been set.

I would go with option 2 and I would have two constructors. One is internal that’s used by the tests where a rand.Rand can be passed in. The exported constructor would not take a rand.Rand, make one, and use the internal constructor. This keeps it so your test (if in the same package) can set a predetermined rand.Rand but the caller doesn’t get to set anything.

There’s no way to guarantee shiny isn’t changed in some code path internally. Go doesn’t have the concept of const or readonly. So basically you just have to make sure nothing sets it outside the constructor

Agree.

For bonus points, don't take rand.Rand but rather func() int. This will make the unit tests you are writing a lot more bearable.

# ? Sep 5, 2021 05:59

Fluue: Jan 2, 2008

Thank you both! Good suggestion to use an internal constructor.

# ? Sep 5, 2021 16:51

Jamus: Feb 10, 2007

Be careful about the logic of setting shiny if it's nil. If you have two threads enter that function at the same time (e.g., if the struct is being read and therefore shiny is being initialized once by two different go-routines) you can end up with two different values of shiny, as if both check if shiny is nil at the same time they'll both go on to randomly generate a value.

A constructor will help to prevent this as you're less likely to accidentally use a constructor in this way, but another solution (if the constructor ends up being un-ergonomic and you want to stay lazy) is the neat sync.Once mutex, which I think is fairly idiomatic. Just set it inside the sync.Once.Do, and then read it directly in the getter. No comparing to nil needed so it works for non-pointer values. This is safe "Because no call to Do returns until the one call to f returns".

Either way, n-thing what the other posters have said, make sure your source of randomness is injectable otherwise testing is going to be annoying.

# ? Sep 6, 2021 00:31

Methanar: Sep 26, 2013; by the sex ghost

I want to marshal json into a struct. Given the following json, I've come up with the following struct. But the keys set for tags could actually be anything, not restricted to name or aggregatedBy. How do I represent this in my struct?

code:

[
  {
    "target": "sumSeries(https_metric.*)",
    "tags": {
      "name": "sumSeries(https_metric.*)",
      "aggregatedBy": "sum"
    },
    "datapoints": [
      [
         null,
        1630946310
      ],
      [
        10,
        1630946320
      ]
    ]
  }
]

code:

type graphiteResults []struct {
	Target string `json:"target"`
	Tags   struct {
		Name         string `json:"name"`
		AggregatedBy string `json:"aggregatedBy"`
	} `json:"tags"`
	Datapoints [][]interface{} `json:"datapoints"`
}

Is this what I want?

code:

type graphiteResults[]struct {
	Target string `json:"target"`
	Tags   map[string]interface{} `json:"tags"`
	Datapoints [][]interface{} `json:"datapoints"`
}

Methanar fucked around with this message at 18:40 on Sep 6, 2021

# ? Sep 6, 2021 18:31

Da Mott Man: Aug 3, 2012

Methanar posted:

I want to marshal json into a struct. Given the following json, I've come up with the following struct. But the keys set for tags could actually be anything, not restricted to name or aggregatedBy. How do I represent this in my struct?
code:
[
  {
    "target": "sumSeries(https_metric.*)",
    "tags": {
      "name": "sumSeries(https_metric.*)",
      "aggregatedBy": "sum"
    },
    "datapoints": [
      [
         null,
        1630946310
      ],
      [
        10,
        1630946320
      ]
    ]
  }
]
code:
type graphiteResults []struct {
	Target string `json:"target"`
	Tags   struct {
		Name         string `json:"name"`
		AggregatedBy string `json:"aggregatedBy"`
	} `json:"tags"`
	Datapoints [][]interface{} `json:"datapoints"`
}

This will handle arbitrary json fields if you marshal the extra data in tags to that member.

Go code:

type graphiteResults []struct {
	Target string `json:"target"`
	Tags   struct {
		Name         string `json:"name"`
		AggregatedBy string `json:"aggregatedBy"`
                Extra map[string]interface{} `json:"-"`
	} `json:"tags"`
	Datapoints [][]interface{} `json:"datapoints"`
}

# ? Sep 6, 2021 18:41

Methanar: Sep 26, 2013; by the sex ghost

So the existence of tags at all is optional. But if it does exist, it's a map of strings with values of also strings.

code:

type graphiteResults[]struct {
	Target string `json:"target"`
	Tags   map[string]interface{} `json:"tags"`
	Datapoints [][]interface{} `json:"datapoints"`
}

# ? Sep 6, 2021 18:44

Da Mott Man: Aug 3, 2012

Methanar posted:

So the existence of tags at all is optional. But if it does exist, it's a map of strings with values of also strings.
code:
type graphiteResults[]struct {
	Target string `json:"target"`
	Tags   map[string]interface{} `json:"tags"`
	Datapoints [][]interface{} `json:"datapoints"`
}

Yeah that should work.

Edit:
Actually I think using anonymous structs would work better if Tags is omitted, the other way if Tags always exists but can be empty.

Go code:

type graphiteResults []struct {
	Target     string          `json:"target"`
	Datapoints [][]interface{} `json:"datapoints"`
}

type OptionalTags graphiteResults

func (results graphiteResults) MarshalJSON() ([]byte, error) {
	return json.Marshal(struct {
		OptionalTags
		Tags map[string]interface{}
	}{
		OptionalTags: OptionalTags(results),
		Tags:         map[string]interface{}{"Name": "testing"},
	})
}

Da Mott Man fucked around with this message at 19:19 on Sep 6, 2021

# ? Sep 6, 2021 19:02

30.5 Days: Nov 19, 2006

go is fuckin good

# ? Apr 6, 2022 00:26

cruft: Oct 25, 2007

30.5 Days posted:

go is fuckin good

# ? Apr 6, 2022 00:52

ErIog: Jul 11, 2001

30.5 Days posted:

go is fuckin good

It is OP, it is. I just doubled my salary (not hard, I was making dog poo poo before, lol) based primarily on me being proficient in Go. I was in academia before doing scaled web services for outside researchers, and I've now gone to the private sector.

The thing that will jokerfy me, though, is that they were real excited to have someone like me on their team during early interviews. Then in final round they dropped the bomb that they don't even use Go, really. They just let the engineers freehand it.

Thankfully my time in academia has prepared me to walk into a mess so I'm kind of excited either way. I like Go because I specifically have a lot of experience with other languages/stacks where things are quite a lot more broken. So it's actually really weird how good their job posting was because it was like exactly suited to someone like me who can come in to clean up messes.

ErIog fucked around with this message at 02:49 on Apr 15, 2022

# ? Apr 15, 2022 02:44

Hughmoris: Apr 21, 2007; Let's go to the abyss!

For you Go'ers (or whatever you call yourselves...): do you use Go for any sort of data analytics / data analyst jobs? I keep seeing it pop up on those type of job postings, and can't tell if I'm missing a big chunk of the data sector or if they are just spitballing keywords for the job description.

# ? Apr 15, 2022 03:42

Breaking Glass: Dec 15, 2021

Hughmoris posted:

For you Go'ers (or whatever you call yourselves...): do you use Go for any sort of data analytics / data analyst jobs?

We use it for logs and event analysis of pretty egregiously large streaming datasets (petabytes). I think Segment does too for data, as well as Uber for lots of different things. It's pretty well suited for getting most of the c++ performance with fewer footguns.

pprof is your friend.

# ? Apr 15, 2022 03:46

Pham Nuwen: Oct 30, 2010

Breaking Glass posted:

We use it for logs and event analysis of pretty egregiously large streaming datasets (petabytes). I think Segment does too for data, as well as Uber for lots of different things. It's pretty well suited for getting most of the c++ performance with fewer footguns.

pprof is your friend.

Funny, I also use it for logs & event analysis. Do you work on an analysis product, and if so, do you mind sharing which one? If not, that's obviously fine.

# ? Apr 15, 2022 03:52

Hughmoris: Apr 21, 2007; Let's go to the abyss!

Breaking Glass posted:

We use it for logs and event analysis of pretty egregiously large streaming datasets (petabytes). I think Segment does too for data, as well as Uber for lots of different things. It's pretty well suited for getting most of the c++ performance with fewer footguns.

pprof is your friend.

That's good to know, thank you. My limited experience has been in healthcare with Python and SQL but I'm starting to look in a new sector and I was puzzled by Go showing up.

# ? Apr 15, 2022 03:58

Adbot: ADBOT LOVES YOU

# ? Apr 18, 2024 11:15

Breaking Glass: Dec 15, 2021

Pham Nuwen posted:

Funny, I also use it for logs & event analysis. Do you work on an analysis product, and if so, do you mind sharing which one? If not, that's obviously fine.

Can't post it here but happy to nerd out over discord if you wanna DM

# ? Apr 15, 2022 04:00

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > The Cavern of COBOL > Go: Google's Plan9 refugees are staging a breakout

«‹›14 »