Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Locked thread
Mniot
May 22, 2003
Not the one you know
I think it's valuable even to people who don't intend to use Erlang, because MononcQc does a nice job of characterizing the problems that appear once you scale beyond a few hundred users and the ways to solve them. A good read.

Adbot
ADBOT LOVES YOU

bobthenameless
Jun 20, 2005

leper khan posted:

I'd be happy to review it. I don't have any production experience with erlang though; I've just used it for hobby projects for a while.

Same here, I've been halfassedly looking around for an "erlang in production" read to prepare myself for a theoretical release of a hobby project.

hit me up on synirc/pm

Are there any other recommended reads on erlang in production?

MononcQc
May 29, 2007

Hi everyone,

Stuff Goes Bad: Erlang in Anger is now available!

Blog post at:
http://engineering.heroku.com/blogs/2014-09-17-erlang-in-anger

Download at:
http://www.erlang-in-anger.com/

Thanks for everyone who reviewed it!

more like dICK
Feb 15, 2010

This is inevitable.
That is extremely cool, thanks.

Mniot
May 22, 2003
Not the one you know
Anyone played with Docker at all? I never got comfortable with relx stuff, but I've been playing around with containers as a way to skip all that.

I make a Dockerfile like
code:
FROM correl/erlang:17.1

ADD ./erlang.mk /opt/app/erlang.mk
ADD ./Makefile /opt/app/Makefile
WORKDIR /opt/app
RUN make deps

ADD . /opt/app
RUN make app

CMD erl -pa ebin -pa deps/*/ebin \
         -noinput -config app.config \
         -eval 'application:ensure_all_started(my_app).'
and things seem to work great. Hopefully I'm not missing something terrible, because this is pretty fun and easy to work with.

MononcQc
May 29, 2007

Well 17.1 has a broken SSL implementation iirc. 17.4 is the latest good stable one that you should move to. 17.1 was broken, 17.3 was also broken. 17.2 was an internal release. So hop on 17.4 if you can.

Never been a fan of erlang.mk personally.

I've started work along with Tristan Sloughter (_tristan3) on rebar3, which we hope to make easier to work with than alternative. like you can go `rebar3 new app myappname` and `rebar3 release` and it should be able to make it all work for you. It's still in pre-alpha state as we break things and add features (profiles, namespaces, tests, etc.) but yeah.

(Also thanks for keeping this thread from disappearing :3:)

Mniot
May 22, 2003
Not the one you know

MononcQc posted:

Never been a fan of erlang.mk personally.

I've started work along with Tristan Sloughter (_tristan3) on rebar3, which we hope to make easier to work with than alternative.

I like Rebar's configuration, but my development cycle with this setup involves rebuilding the container when code's modified, and `rebar compile skip_deps=true` is crazy-slow compared to `make app` with erlang.mk. I'm not sure why that would be...

Edit: 17.3 is totally broken (like, it crashes if you try to make an TLS connection). 17.1 can connect, at least. I'll give 17.4 a shot.

Mniot fucked around with this message at 14:56 on Dec 31, 2014

MononcQc
May 29, 2007

Mniot posted:

I like Rebar's configuration, but my development cycle with this setup involves rebuilding the container when code's modified, and `rebar compile skip_deps=true` is crazy-slow compared to `make app` with erlang.mk. I'm not sure why that would be...

`rebar compile skip_deps=true` will remain necessary on rebar because everyone has documented their stuff as 'rebar get-deps compile' to get going and we couldn't break that. In rebar3, 'rebar compile' will fetch deps and recompile those that need recompiling only, and will use the lock file shenanigans. It will therefore generally skip deps and be faster than what you have in rebar by default, but probably won't be as fast as erlang.mk.

On the other hand, you won't be using hand-patched makefiles, and it will do conflict resolution (maven-style -- top dependency wins) while warning you when a dep has been overriden. Profiles are cool too --you can have test-only dependencies or lighter-weight releases when not in the 'prod' profile, which used to be a huge pain in the rear end to do.

In any case, rebar is usually a lot faster in linux than it is on OSX for some reason I don't know, like five times slower or something, and rebar3, again, is still alpha.

While I can't deny the speed of erlang.mk, I can't wrap my head around the idea of "hm yes I'd love to move more of my build stuff to makefiles", as much as I also hate the idea that "let's have everyone ship their build system with their libs" that rebar uses.

Mniot
May 22, 2003
Not the one you know

MononcQc posted:

On the other hand, you won't be using hand-patched makefiles, and it will do conflict resolution (maven-style -- top dependency wins)

Maven's an interesting comparison. I really enjoyed having Maven take care of my dependencies, but if I had to do `mvn package` to see anything I'd get pretty cranky. But Eclipse's incremental compiler is snappy and so I'd code and test with that and only fire up Maven when I was done writing.

Your "How I Start" post seem to say that you just don't test your code until you're done writing. I really wish I had something recompiling and reloading my code for me behind the scenes.

MononcQc posted:

While I can't deny the speed of erlang.mk, I can't wrap my head around the idea of "hm yes I'd love to move more of my build stuff to makefiles",

Well maybe you haven't spent hundreds of hours in grad school making gMake do awful things. :) Makefiles are incredibly versatile: an Ops guy once told me he does all his shell scripting in Make rather than Bash: no screwing with "set -e", sane command-echoing, and you can always put bash inside your Make so you don't lose any power at all.

It's not as pretty, but it's a lot more hackable. For example, I got some libraries that do Rebar but not erlang.mk, and they were compiling their sub-dependencies down in sub-sub-directories. Rather than figure out how to do it cleanly, I added this rule to my makefile:

code:
link:
	find $(PWD)/deps -name \*.beam -exec ln -s '{}' $(PWD)/ebin/ \;
	find $(PWD)/deps -name \*.app -exec ln -s '{}' $(PWD)/ebin/ \;
That symlinks every .beam and .app under the deps/ directory into my ebin/. With a little more work, I could make it a breadth-first traversal to get the top-wins dependency resolution with warnings that you're talking about.

Anyway, not that makefiles are the best, but I find it very easy to wrap my head around "I wish a Makefile was controlling this".

MononcQc
May 29, 2007

Mniot posted:

Your "How I Start" post seem to say that you just don't test your code until you're done writing. I really wish I had something recompiling and reloading my code for me behind the scenes.

That really depends on the project. For a state machine from a known specification and such a small program, I really did my testing in a REPL. There were like 4 states possible and little more than one or two possible transitions between them.

The bigger the project, the faster I am to shift away from the REPL and into a test suite -- there will be more and more behaviours to test in an ever-increasing scope, and REPL sessions rapidly lose their attractiveness in favor of automation for everything.

quote:

Well maybe you haven't spent hundreds of hours in grad school making gMake do awful things. :) Makefiles are incredibly versatile: an Ops guy once told me he does all his shell scripting in Make rather than Bash: no screwing with "set -e", sane command-echoing, and you can always put bash inside your Make so you don't lose any power at all.

Yeah, that's probably it. My skill with makefiles limits me to wrapping existing tools in ad-hoc ways that may lack portability and are hard to test and maintain. I don't have tooling around it to make for proper plugins, help, integration, whatever -- for me it's all hacks. We've made big changes in rebar3 regarding plugins: giving them a common interface, adding in namespaces to create 'suites' (so you can do 'rebar3 lfe compile' and compile non-Erlang files with a custom implementation, for example), normalizing for templates, etc.

Adding this stuff around a makefile sounds like a lot of hurt. We still haven't handled C stuff, but there's thinking about just having it call a make file rather than trying to do C Port compiling in Erlang, which is now suddenly a very bad toolchain for that.

Mniot
May 22, 2003
Not the one you know

MononcQc posted:

That really depends on the project. For a state machine from a known specification and such a small program, I really did my testing in a REPL. There were like 4 states possible and little more than one or two possible transitions between them.

The bigger the project, the faster I am to shift away from the REPL and into a test suite -- there will be more and more behaviours to test in an ever-increasing scope, and REPL sessions rapidly lose their attractiveness in favor of automation for everything.

I think a big part of it is that I'm not experienced enough with Erlang to write for long stretches. I'm generally going "Does this call returns a binary or a list? What can I match against to pull out that username? Will this function do what I'm expecting?" And I get there by compiling all my code, starting 'erl', running 'application:ensure_all_started(my_app).', and then poking around. But every time I need to change something (like my Cowboy handler needs a callback added to it), I exit the REPL and start again.

MononcQc
May 29, 2007

You can just call l(my_handler) after the thing was compiled too and avoid having to reset everything.

EDIT: there's also some cool autoreloaders around (https://github.com/rustyio/sync), but I can't really vouch for them because I was always too lazy to set them up.

MononcQc fucked around with this message at 21:27 on Dec 31, 2014

MononcQc
May 29, 2007

Huge 3 month bump to announce Rebar3 Alpha is now out -- http://lists.basho.com/pipermail/rebar_lists.basho.com/2015-March/002137.html Myself, coworkers at Heroku, and contributors (some from here -- talentdeficit) have been working on upgrading rebar into something more usable. It's still likely buggy, but should be stable enough.

People on here who want to try Erlang could give rebar3 a spin and leave some comments if they feel like so. See the basic usage docs. It's an end-to-end build tool, and some people might dislike that, but oh well.

Serenade
Nov 5, 2011

"I should really learn to fucking read"
I want to learn and play around with Erlang by making a library to call from the Unreal Engine. Is there a way built into Erlang to use C or C++ as a shallow wrapper and do most of my code in Erlang? I've found C nodes and native implemented functions, but those both go the wrong direction.

I think I could do it with pipes in C nodes but I'd rather a different path.

bobthenameless
Jun 20, 2005

As far as I know, if you're wanting to do that probably the best way would be through a socket; but you have to consider that any erlang library you write would need to be running within the Erlang VM - it wouldn't be a compiled dll you can just use from within UE4 without also having the VM going.

I might be wrong about that, I'm far from experienced with erlang, and when I was looking at it a few months back i was looking going the ways you already mentioned.

Maybe there is a way to bundle it all into one thing; I just haven't run across anywhere talking about that.

Serenade
Nov 5, 2011

"I should really learn to fucking read"

bobthenameless posted:

As far as I know, if you're wanting to do that probably the best way would be through a socket; but you have to consider that any erlang library you write would need to be running within the Erlang VM - it wouldn't be a compiled dll you can just use from within UE4 without also having the VM going.

I might be wrong about that, I'm far from experienced with erlang, and when I was looking at it a few months back i was looking going the ways you already mentioned.

Maybe there is a way to bundle it all into one thing; I just haven't run across anywhere talking about that.

If that has to be done I think that'll still be manageable. I could just start the VM in an init C function and close it on exit or UE crash.

MononcQc
May 29, 2007

Yeah I haven't really heard of anyone booting an Erlang VM from C except in one case, where the point was to get an Erlang VM to be replaced by a different version of the VM (see https://github.com/tsloughter/rebar3_run).

Generally the tip seems to be "don't do that, just call C from Erlang", which yeah, doesn't work much for you.

Mniot
May 22, 2003
Not the one you know
I did some LISP/C IPC a while back and found the easiest thing was to start out in Lisp and fork the C code as a child process, even though the two programs were conceptually peers. Lisp went first because I was running the Lisp code inside an interpreter, so the launch process was complex while the C code was a statically linked executable.

Doing C/Erlang communication over a socket should be fun and easy. Erlang's binary matching means you can just send and receive packed binary structs in C. (I think this is not actually kosher, but if you only use one compiler it's great.)

Serenade
Nov 5, 2011

"I should really learn to fucking read"
I looked into it further, for Unreal 3 at least, the only way I saw it attempted was be running Unreal from the vm but that leads to very poor framerate for a game.

Oh well, I'll have to find some other project that allows me to both learn erlang and prove it if I want a relevant job.

the talent deficit
Dec 20, 2003

self-deprecation is a very british trait, and problems can arise when the british attempt to do so with a foreign culture





Serenade posted:

I looked into it further, for Unreal 3 at least, the only way I saw it attempted was be running Unreal from the vm but that leads to very poor framerate for a game.

Oh well, I'll have to find some other project that allows me to both learn erlang and prove it if I want a relevant job.

can you not just have a client-server architecture where the erlang program runs alongside unreal 3?

Adbot
ADBOT LOVES YOU

Mniot
May 22, 2003
Not the one you know
I've been using rebar3 for some new projects and really enjoying it. The speed feels way up from the last time I tried it (or rebar) and it's doing a pretty great job of building things the way I expect. "rebar3 eunit" and "rebar3 dialyzer" both work well, and it makes it really easy and fun to write my code with tests and type-specs. The deps system (where there's a "rebar.lock" that pins itself to a specific commit, and you have to "rebar3 upgrade" if you want to change it) took some getting used to, but I'm pretty happy with it in my workflow now. Thanks for working on it!

I've also been using Spacemacs with EDTS enabled. It's not quite perfect as a rebar3-aware flymake, but it's really close. If you like Emacs for Erlang but aren't an Emacs pro, I recommend checking Spacemacs out.

  • Locked thread