Register a SA Forums Account here!
JOINING THE SA FORUMS WILL REMOVE THIS BIG AD, THE ANNOYING UNDERLINED ADS, AND STUPID INTERSTITIAL ADS!!!

You can: log in, read the tech support FAQ, or request your lost password. This dumb message (and those ads) will appear on every screen until you register! Get rid of this crap by registering your own SA Forums Account and joining roughly 150,000 Goons, for the one-time price of $9.95! We charge money because it costs us money per month for bills, and since we don't believe in showing ads to our users, we try to make the money back through forum registrations.
 
  • Post
  • Reply
the talent deficit
Dec 20, 2003

self-deprecation is a very british trait, and problems can arise when the british attempt to do so with a foreign culture





Boz0r posted:

What are peoples' thoughts on TeamCity vs Jenkins?

awful vs horrid

chasing good build servers is a waste of time: there isn't one and you should stick with what you have

Adbot
ADBOT LOVES YOU

Bhodi
Dec 9, 2007

Oh, it's just a cat.
Pillbug
Jenkins works for me but I wouldn't call it enthusiastic approval.

It's a pita to wrangle though and for something that enables continuous integration, saving/storing/updating it's own state is a real PITA.

Vulture Culture
Jul 14, 2003

I was never enjoying it. I only eat it for the nutrients.
TeamCity is a great build server but nothing is very good at handling the deploy end of CD without a ton of duct tape and glue. That said, I've almost never run into weird operational problems with it, unlike Jenkins.

New Yorp New Yorp
Jul 18, 2003

Only in Kenya.
Pillbug

Vulture Culture posted:

TeamCity is a great build server but nothing is very good at handling the deploy end of CD without a ton of duct tape and glue. That said, I've almost never run into weird operational problems with it, unlike Jenkins.

That's why the deployment piece is being foisted off onto configuration management systems for the most part. Overextending a build system to do deployments sucks. Plus builds pushing bits encourages building per environment instead of promoting changes from one environment to the next.

Boz0r
Sep 7, 2006
The Rocketship in action.

the talent deficit posted:

you should stick with what you have

What I have is four developers trying to do weird patch jobs for each deploy. I think anything would be better than that.

Vulture Culture
Jul 14, 2003

I was never enjoying it. I only eat it for the nutrients.

Ithaqua posted:

That's why the deployment piece is being foisted off onto configuration management systems for the most part. Overextending a build system to do deployments sucks. Plus builds pushing bits encourages building per environment instead of promoting changes from one environment to the next.
While they're extremely useful for solving some of the problems around drift that hinder code deploys, configuration management doesn't really solve (m)any deployment problems for anything except really trivial cases, though. Automating the CD pipeline to any sane degree (that is, without wads of tape) requires insane cognitive investments in stuff like service discovery and container schedulers (or virtual machine images, if that's your thing). Even commercial solutions around Docker like EC2 Container Service or Google Container Engine are really warty and weird, and at absolute best there's no standardization between ZK, etcd, Consul/Serf, and their kin when it comes to actually communicating and messaging between your service instances. Blue/green deploys as handled by some of these systems are pretty cool, but can't effectively handle patterns like swapping out messaging or service discovery systems between deployed versions.

The product I'm managing is sitting somewhere between 3,000 and 4,000 service instances now, and there's nothing yet that operates at that scale and is understandable by normal humans.

Vulture Culture fucked around with this message at 21:45 on Feb 2, 2016

the talent deficit
Dec 20, 2003

self-deprecation is a very british trait, and problems can arise when the british attempt to do so with a foreign culture





Boz0r posted:

What I have is four developers trying to do weird patch jobs for each deploy. I think anything would be better than that.

if deploy is your problem (and not building) i would encourage you to look elsewhere. aws codedeploy is pretty good if you are on aws

Vulture Culture
Jul 14, 2003

I was never enjoying it. I only eat it for the nutrients.

the talent deficit posted:

if deploy is your problem (and not building) i would encourage you to look elsewhere. aws codedeploy is pretty good if you are on aws
If they have no intention of ever properly supporting WebSockets on ELB it would be nice if their deploy tooling would at least support another load balancer though

Boz0r
Sep 7, 2006
The Rocketship in action.
After further examination of the problem, we actually just want to a build server that can do tests on the build before the code can be sent to the version control server. And just have a button on the build server that spits out the newest version, that we'll manually deploy.

New Yorp New Yorp
Jul 18, 2003

Only in Kenya.
Pillbug

Boz0r posted:

After further examination of the problem, we actually just want to a build server that can do tests on the build before the code can be sent to the version control server. And just have a button on the build server that spits out the newest version, that we'll manually deploy.

So you're looking for (in centralized VC terms) a gated build, or in DVCS terms a pull request + build policy?

Boz0r
Sep 7, 2006
The Rocketship in action.
I don't know what it's called, I just want to be able to click a button in IntelliJ that sends the code to TeamCity or whatever. That builds it, runs all the tests and verifies policies and, if it's supergreen, passes the code along to SVN.

EDIT: I think JetBrains calls it a delayed commit.

New Yorp New Yorp
Jul 18, 2003

Only in Kenya.
Pillbug

Boz0r posted:

I don't know what it's called, I just want to be able to click a button in IntelliJ that sends the code to TeamCity or whatever. That builds it, runs all the tests and verifies policies and, if it's supergreen, passes the code along to SVN.

EDIT: I think JetBrains calls it a delayed commit.

Yeah, TFS/VSTeam Services has something called a gated checkin that accomplishes that for their CVCS. Their new build system can build from Subversion, but they haven't implemented gated checkin yet.

For Git, they have pull requests + branch policies, where it will reject the pull request automatically if a build fails.

YO MAMA HEAD
Sep 11, 2007

I only really focus on devopsy stuff for a few days a month (or when something breaks) but I was surprised I hadn't heard anything at all about Otto. Has anyone checked it out? On the dev side I like the idea of a simpler Appfile but I didn't understand provisioning—if we need a dev box with requirements that are in any way out of the ordinary (SoX for audio processing), how do we make that happen?

I didn't even explore any of the deployment functionality etc. but for some of our simpler projects it would be nice to not have to mess with a Bamboo pipeline.

Hughlander
May 11, 2005

Ithaqua posted:

For Git, they have pull requests + branch policies, where it will reject the pull request automatically if a build fails.

We have something like that using the pre receive hook of git. If a commit comes in and it hasn't passed tests schedule them. If it's master and tests hasn't run/passed. Reject the push.

Vulture Culture
Jul 14, 2003

I was never enjoying it. I only eat it for the nutrients.

YO MAMA HEAD posted:

I only really focus on devopsy stuff for a few days a month (or when something breaks) but I was surprised I hadn't heard anything at all about Otto. Has anyone checked it out? On the dev side I like the idea of a simpler Appfile but I didn't understand provisioning—if we need a dev box with requirements that are in any way out of the ordinary (SoX for audio processing), how do we make that happen?

I didn't even explore any of the deployment functionality etc. but for some of our simpler projects it would be nice to not have to mess with a Bamboo pipeline.
Otto's nice if you need to integrate a deploy process with Terraform. If you're not already using Terraform, it's total overkill.

sunaurus
Feb 13, 2012

Oh great, another bookah.
I've been using self-hosted Bamboo (+ JIRA) for a year for my personal projects. The starter license is not that expensive, but the 10 build jobs limit has been very annoying. I was wondering, before I renew my license, is there something better that I could use in the same price range or for free?

My situation is basically just around 5-10 active projects with private repos (most of them only have one or two committers) - most of them only get built and deployed <10 times per month. I would love any alternatives to be self-hosted as well, and integration with an issue tracker (doesn't have to be JIRA) would be amazing. Is anybody in a similar boat? What are you guys using?

sunaurus fucked around with this message at 11:13 on Feb 6, 2016

New Yorp New Yorp
Jul 18, 2003

Only in Kenya.
Pillbug

Illegal Move posted:

I've been using self-hosted Bamboo (+ JIRA) for a year for my personal projects. The starter license is not that expensive, but the 10 build jobs limit has been very annoying. I was wondering, before I renew my license, is there something better that I could use in the same price range or for free?

My situation is basically just around 5-10 active projects with private repos (most of them only have one or two committers) - most of them only get built and deployed <10 times per month. I would love any alternatives to be self-hosted as well, and integration with an issue tracker (doesn't have to be JIRA) would be amazing. Is anybody in a similar boat? What are you guys using?

Team Foundation Server Express. On-premise installation, free for up to 5 people. Full work tracking / source control / build.

sunaurus
Feb 13, 2012

Oh great, another bookah.

Ithaqua posted:

Team Foundation Server Express. On-premise installation, free for up to 5 people. Full work tracking / source control / build.

Thanks, that sounds great, but from what I can tell, it only runs on windows? Sadly, I don't have any windows boxes.
From what I know, ssh from windows is a huge pain, so I'm assuming that even if I set up a virtual machine for this, actual deployments to my (linux) servers would not be easy to set up?

New Yorp New Yorp
Jul 18, 2003

Only in Kenya.
Pillbug

Illegal Move posted:

Thanks, that sounds great, but from what I can tell, it only runs on windows? Sadly, I don't have any windows boxes.
From what I know, ssh from windows is a huge pain, so I'm assuming that even if I set up a virtual machine for this, actual deployments to my (linux) servers would not be easy to set up?

Correct, the actual server piece runs on Windows. The build and release infrastructure is cross-platform, however -- the build/release agent is node.js.

It's worth looking at Team Services. Same restrictions (free for up to 5 people) but you don't have to worry about the infrastructure side. Build is still free assuming you set up an on-prem agent.

Cancelbot
Nov 22, 2006

Canceling spam since 1928

I want to beef up my skills as I branch out more from .NET stuff. Currently I work with TeamCity + Octopus a lot in my current role but I want to learn something more powerful and generic, like Chef or Ansible. Where is a good place to start and is there a way I can play around with this stuff easily?

The whole immutable server thing has resonated with me and would love to build something that does a rolling deploy that destroys the "old-live" once "new-live" is up, even if I'm just scripting some private AWS instances.

Cancelbot fucked around with this message at 00:23 on Feb 9, 2016

Sedro
Dec 31, 2008
I'm looking to separate my build server from my deployment. Currently TeamCity builds a feature branch then runs a shell script to start an environment locally (passing in the branch name). Instead I want to push the artifact to a QA server and deal with it there. A web interface to manage those environments would be nice too. What should I be looking for (besides a proper devops engineer)?

This is for the development team, not hosting customers. The build artifact is a set of docker images.

Vulture Culture
Jul 14, 2003

I was never enjoying it. I only eat it for the nutrients.

Sedro posted:

I'm looking to separate my build server from my deployment. Currently TeamCity builds a feature branch then runs a shell script to start an environment locally (passing in the branch name). Instead I want to push the artifact to a QA server and deal with it there. A web interface to manage those environments would be nice too. What should I be looking for (besides a proper devops engineer)?

This is for the development team, not hosting customers. The build artifact is a set of docker images.
Do exactly what you're doing now (maybe replace your shell script with some Compose configs if you need to), but point your docker client at a different server

Erwin
Feb 17, 2006

Anybody know when Policyfiles are going to be considered not experimental/ready for use in ChefDK? I'm trying to improve my cookbook workflow but I don't want to get too invested in Berkshelf if it's going away soon.

Vulture Culture
Jul 14, 2003

I was never enjoying it. I only eat it for the nutrients.

Erwin posted:

Anybody know when Policyfiles are going to be considered not experimental/ready for use in ChefDK? I'm trying to improve my cookbook workflow but I don't want to get too invested in Berkshelf if it's going away soon.
I got so loving sick of both these options that I manage my dependency versioning by hand now, it's really not that hard. I run berks-api on my Chef server for Test Kitchen basically but berks never touches production

Erwin
Feb 17, 2006

As in adding dependencies to metadata and pinning versions to environments? That's what I'm doing now and I'm fine with continuing to do that, but Berkshelf being in the ChefDK seemed to indicate that that's the way it's meant to be done. But in general, the 'way to do things' in Chef changes every 3 months anyway.

Vulture Culture
Jul 14, 2003

I was never enjoying it. I only eat it for the nutrients.
It's a really very valuable tool for resolving dependency chains in test suites, especially if you're targeting local forks of things, but the "conventional wisdom" in Chef is based around things that work for teams of 20 Chef-focused engineers and are absolutely irrelevant for small teams.

We're in the process of rapidly using Chef to manage less and less things anyway, so it's becoming less relevant to us in any case. CM systems like Chef and Puppet are the God Object development anti-pattern applied to infrastructures. Abstractions aren't ever as fixed as you think they are, and we just spend more time unspinning and respinning balls of yarn than we do getting work done with the tool. As we move towards considerably-less-mutable infrastructures, idempotence is just another added cost. I'm feeling much less stressed with a big pile of Dockerfiles, honestly.

Vulture Culture fucked around with this message at 19:28 on Feb 10, 2016

wwb
Aug 17, 2004

For chef I'm still somewhat in favor of manual dependency resolution if for no other reason than it helps you keep you mindful as to what you are running. Some of those major cookbooks are way, way bloated and have a bunch of odd dependencies that can bite you in the rear end down the road.

I'll also note that, IMHO, Chef/Ansible really do different things than one should be doing w/ TeamCity and Octopus. In our shop we divide responsibility for spinning up machines (chef) and deploying apps (TeamCity). YMMV.

Swink
Apr 18, 2006
Left Side <--- Many Whelps
I just landed a new role at a software company. I'll be learning/doing build automation. They're a .Net shop and use the Atlassian suite.

I'd like to do a bit of cramming on the subject before I start as I have zero experience in software. Can anyone suggest any books to read, blogs to follow or whatever to help me immerse myself in this area?

I understand the basics, but I'm sure there's a big dif between what happened in The Phoenix Project and what happens in the real world.

Virigoth
Apr 28, 2009

Corona rules everything around me
C.R.E.A.M. get the virus
In the ICU y'all......



Has anyone here used Spinnaker and has thoughts on it? Our system is a Jenkins master right now that supports 3 lower environments for dev, QA, and our staging environment and then 4 production regions in AWS. We manage around 2,000 servers for micro-services but are growing that number daily it seems. Our Jenkins master does about 16,000 jobs a day and uses a in-house jslave script to make sure we have enough executors in each environment. With the permissions needed to deploy this is a scary scenario. What I'm considering doing a proof of concept on is hiding Jenkins behind Spinnaker and using it just like the demo video sets up with Spinnaker being our front end and a lot of user access removed.

Blinkz0rz
May 27, 2001

MY CONTEMPT FOR MY OWN EMPLOYEES IS ONLY MATCHED BY MY LOVE FOR TOM BRADY'S SWEATY MAGA BALLS

Virigoth posted:

Has anyone here used Spinnaker and has thoughts on it? Our system is a Jenkins master right now that supports 3 lower environments for dev, QA, and our staging environment and then 4 production regions in AWS. We manage around 2,000 servers for micro-services but are growing that number daily it seems. Our Jenkins master does about 16,000 jobs a day and uses a in-house jslave script to make sure we have enough executors in each environment. With the permissions needed to deploy this is a scary scenario. What I'm considering doing a proof of concept on is hiding Jenkins behind Spinnaker and using it just like the demo video sets up with Spinnaker being our front end and a lot of user access removed.

Make sure you post how it goes. We're evaluating Spinnaker but we're still not entirely sure where to drop it into our pipeline and how to use it to replace all the stuff we use in Asgard.

Pixelboy
Sep 13, 2005

Now, I know what you're thinking...

sunaurus posted:

From what I know, ssh from windows is a huge pain, so I'm assuming that even if I set up a virtual machine for this, actual deployments to my (linux) servers would not be easy to set up?

Not sure why you think this is true... I've never had a problem w/ ssh. Unless you're posting from 1996.

wwb
Aug 17, 2004

SSH from windows is pretty easy these days.

Microsoft is working on an official solution, but for now you can just go download portable git and add it's bin folder to your path -- that will get you cross-compiled ports of most of the *nix toolchain including ssh.exe.

Thermopyle
Jul 1, 2003

...the stupid are cocksure while the intelligent are full of doubt. —Bertrand Russell

Install Linux putty, problem solved.

Vulture Culture
Jul 14, 2003

I was never enjoying it. I only eat it for the nutrients.

wwb posted:

SSH from windows is pretty easy these days.

Microsoft is working on an official solution, but for now you can just go download portable git and add it's bin folder to your path -- that will get you cross-compiled ports of most of the *nix toolchain including ssh.exe.
msys2+pacman is super easy to use these days too.

stubblyhead
Sep 13, 2007

That is treason, Johnny!

Fun Shoe
git has an SSH client as well as a few other unix utilities, though I don't think the directory gets added to your path when you install.

Fcdts26
Mar 18, 2009
Xshell is an awesome way to do SSH in windows.

revmoo
May 25, 2006

#basta
So we're moving from Bamboo and stash to Gitlab.

I just setup Gitlab-ci and pushed a successful test deploy. Then I realized that there's absolutely no interface at all for managing builds and environements. What the gently caress?

I get that I can set up tagged releases for prod pushes and use branches to dictate dev server pushes, but this is a HUGE pain in the rear end. We normally push builds around to various test/staging/beta/prod servers willy nilly and not having an interface to do this is likely to make my pull the plug and use Jenkins or something instead.

Am I missing something or does Gitlab-CI really have no interface at all for managing deployments?

Pollyanna
Mar 5, 2005

Milk's on them.


We're actually in the middle of implementing CICD for our current apps. So far, we're using a combination of Vagrant, Docker, Jenkins, Make, and a homegrown Elastic Beanstalk tool for building and deployment :shepface: One majorly important thing I've learned is not to go too far away from established/pre-existing solutions developed by other members of your team(s), because having everyone converge on one specific CICD solution is incredibly important. Otherwise, nobody knows what the gently caress.

It does help a gently caress of a lot not having to use Chef or anything for provisioning, and instead just using Vagrant boxes and Docker images for that bullshit. Just containerize everything, problem solved! :v:

Hadlock
Nov 9, 2004

We have a fledgling CI system for our Ruby on Rails product, Jenkins feeds Team City from Github, and also we have Bamboo now building our product in containers and deploying it to GCE.

As near as I can tell, Bamboo basically stopped development in 2012, with some bug fixes in 2013 and Atlassian hasn't touched the drat thing in years except for periodic life support bug fixes.

Bamboo will build the docker containers and deploy via shell scripts, but at that point it's just a glorified scheduler with deep hooks in to Github. It also does some testing, but all the results need to be in JUnit XML format to do anything with them

I've found an rspec to JUnit conversion library so... that should be possible.

I'm also responsible for testing the API, and integration testing at the GUI (Selenium) level before it goes out to the customer. I'm thinking of writing some shell scripts to test the API endpoints with some matching criteria, and also some Selnium Grid (dockerized selnium endpoints) monstrosity.

Nobody seems to do Selenium very well, except for it looks like... Travis CI and maaaaybe Sauce Labs? I guess everyone else has their own home-grown widget factory.

So I guess, this is sort of what I'm looking at

Bamboo builds docker container
Bamboo deploys docker container to dev cluster
Bamboo runs testing shell scripts
-Integration/UI test
-API Test
-Performance Test
-Code coverage
-Static analysis

Then most of that goes in to a series of JUnit XML files and dumped back to Bamboo
Data also gets piped in to a data analysis db

Profit? What is the hot new awesome-sauce everyone is using that's not Rainforest QA?

Adbot
ADBOT LOVES YOU

Bhodi
Dec 9, 2007

Oh, it's just a cat.
Pillbug
Honestly, it sounds like you can cut bamboo out of all of that. Jenkins should be able to do all of what it's doing.

I use serverspec + rspec + selenium to do integration testing on our project and I'm using the same "RspecJunitFormatter" converter you probably found on github for jenkins to read.

Right now, it's just a hot mess of applications stacked on top of each other and hotglued with shell scripts just like what you're doing.

  • 1
  • 2
  • 3
  • 4
  • 5
  • Post
  • Reply