I've only ever used cfengine and Ansible, but they are both declarative. Hell, Ansible uses yaml files too.
I would be somewhat surprised to find out Puppet and Chef weren't declarative either. Because setting up a system in an imperative fashion is ripe for trouble. You may as well use bash scripts at that point.
I've used Ansible for close to 10 years for hobby projects. And setting up my development environment. Give me a freshly installed Ubuntu laptop, and I can have my development environment 100% setup with a single command.
Ansible is YAML, but it's definitely imperative YAML - each YAML file is a list of steps to execute. It uses YAML kind of like how Lisp uses S-expressions, as a nice data structure for people to write code in, but it's still code.
Sure, the steps are things like "if X hasn't been done yet, do it." That means it's idempotent imperative code. It doesn't mean it's declarative.
CFEngine is slightly less imperative, but when I was doing heavy CFEngine work I had a printout on my cubicle wall of the "normal ordering" because it was extremely relevant that CFEngine ran each step in a specific order and looped over that order until it converged, and I cared about things like whether a files promise or packages promise executed first so I could depend on one in the other.
Kubernetes - largely because it insists you use containers - doesn't have any concept of "steps". You tell it what you want your deployment to look like and it makes it happen. You simply do not have the ability to say, install this package, then edit this config file, then start this service, then start these five clients. It does make it harder to lift an existing design onto Kubernetes, but it means the result is much more robust. (For some of these things, you can use Dockerfiles, which are in fact imperative steps - but once a build has happened you use the image as an artifact. For other things, you're expected to write your systems so that the order between steps doesn't matter, which is quite a big thing to ask, but it is the only manageable way to automate large-scale deployments. On the flip side, it's overkill for scale-of-one tasks like setting up your development environment on a new laptop.)
I agree. It is impressive how much it can orchestrate. It is also very useless in the real cloud because developers there are dealing with higher-level abstractions to solve problems for the business.
The most simplistic task - execute some code in response to even in a bucket - makes kubernetes with all its sophisticated convergence capabilities completely useless. And even if somebody figures this out and puts the opensource project on github to do this on kubernetes - it just going to break at slightest load.
Not to mention all the work to run kubernetes at any acceptable level of security, or keep the cost down, do all patching, scaling, logging, upgrades... Oh, the configuration management itself for kubernetes? Ah sorry, I forgot, there are 17 great open-source projects exists :)
> The most simplistic task - execute some code in response to even in a bucket - makes kubernetes with all its sophisticated convergence capabilities completely useless.
That's because you're not thinking web^Wcloud scale. To execute some code in response to event you need:
- several workers that will poll the source bucket for changes (of course you could've used existing notification mechanism like aws eventBridge, but that will couple you k8s to vendor-specific infra, so it kinda deminishes the point of k8s)
- distributed message bus with persistanse layer. Kafka will work nicely because they say so on Medium, even though it's not designed for this use case
- a bunch of stateless consumers for the events
- don't forget that you'll need to write processing code with concurrency in mind because you're actually executing it in truly destributed system at this point and you've made a poor choice for your messaging system
Wait, I can do all these with s3 and lambda at any scale - for pennies :) Will probably take few hours to set everything up with tools like stackery.io
So once again, why developers need kubernetes for? If the most simple problem becomes a habitholy mess :)
You can save your Kubernetes manifests in any order. Stuff that depends on other stuff just won't come up until the other stuff exists.
For example, I can declare a Pod that mounts a Secret. If the Secret does not exist, the Pod won't start -- but once I create the Secret the pod will start without requiring further manual intervention.
What Kubernetes really is, under the hood, is a bunch of controllers that are constantly comparing the desired state of the world with the actual state, and taking action if the actual state does not match.
The configuration model exposed to users is declarative. The eventual consistency model means you don't need to tell it what order things need to be done.
A combination of things, mostly related to Kubernetes' scope and use case being different from Ansible/CFEngine/etc. Kubernetes actually runs your environment. Ansible/CFEngine/etc. set up an environment that runs somewhere else.
This is basically the benefit of "containerization" - it's not the containers themselves, it's the constraints they place on the problem space.
Kubernetes gives you limited tools for doing things to container images beyond running a single command - you can run initContainers and health checks, but the model is generally that you start a container from an image, run a command, and exit the container when the command exits. If you want the service to respawn, the whole container respawns. If you want to upgrade it, you delete the container and make a new one, you don't upgrade it in place.
If you want to, say, run a three-node database cluster, an Ansible playbook is likely to go to each machine, configure some apt sources, install a package, copy some auth keys around, create some firewall rules, start up the first database in initialization mode if it's a new deployment, connect the rest of the databases, etc. You can't take this approach in Kubernetes. Your software comes in via a Docker image, which is generated from an imperative Dockerfile (or whatever tool you like), but that happens ahead of time, outside of your running infrastructure. You can't (or shouldn't, at least) download and install software when the container starts up.
You also can't control the order when the containers start up - each DB process must be capable of syncing up with whichever DB instances happen to be running when it starts up. You can have a "controller" (https://kubernetes.io/docs/concepts/architecture/controller/) if you want loops, but a controller isn't really set up to be fully imperative, either. It gets to say, I want to go from here to point B, but it doesn't get much control of the steps to get there. And it has to be able to account for things like one database server disappearing at a random time. It can tell Kubernetes how point B looks different from point A, but that's it.
And since Kubernetes only runs containers, and containers abstract over machines (physical or virtual), it gets to insist that every time it runs some command, it runs in a fresh container. You don't have to have any logic for, how do I handle running the database if a previous version of the database was installed. It's not - you build a new fresh Docker image, and you run the database command in a container from that image. If the command exits, the container goes away, and Kubernetes starts a new container with another attempt to run that command. It can do that because it's not managing systems you provide it, it's managing containers that it creates. If you need to incrementally migrate your data from DB version 1 to 1.1, you can start up some fresh containers running version 1.1, wait for the data to sync, and then shut down version 1 - no in-place upgrades like you'd be tempted to do on full machines.
And yeah, for databases, you need to keep track of persistent storage, but that's explicitly specified in your config. You don't have any problems with configuration drift (a serious problem with large-scale Ansible/CFEngine/etc.) because there's nothing that's unexpectedly stateful. Everything is fully determined by what's specified in the latest version of your manifest because there's no other input to the system beyond that.
Again, the tradeoff is this makes quite a few constraints on your system design. They're all constraints that are long-term better if you're running at a large enough scale, but it's not clear the benefits are worth it for very small projects. I prefer running three-node database clusters on stateful machines, for instance - but the stateless web applications on top can certainly live in Kubernetes, there's no sense caring about "oh we used to run a2enmod but our current playbook doesn't run a2dismod so half our machines have this module by mistake" or whatever.
It is common to have significant logic and complexity in the configuration management manifests, but I'd argue that it's possible to move most of that to packaging and have your configuration management just be "package state latest, service state restarted."
Check out nix for actual development environments. Huge fan of that as well.
I can buy a new laptop and be back to 100% in a few minutes. Though the amount of time I spent learning how to get there far exceeds any time savings. Ever.
I've been testing out nix and I haven't found out how to install packages in a declarative way yet. Using "nix-env -iA <whatever>` seems really imperative.
How are you doing that? Do you use something like home-manager, or do you just define a default.nix and then nix-shell it whenever you need something?
Yes `nix-env -iA` is installing packages in an imperative way. I think it is there to be some kind of tool that people from other OS can relate to. Purist say you should avoid using it for installing packages and instead list global packages in `/etc/nixos/configuration.nix` for globally installed packages and home-manager for user specific ones, and if you need temporarily just to try something out use `nix-shell -p <whatever>`.
Back to your second question, you can configure the system through `/etc/nixos/configuration.nix` it is enough to configure system as a service. Pretty much everything you could do through Chef/Puppet/Saltstack/Ansible/CFEngine etc.
home-manager is taking it a step further and do this kind of configuration per user. It is actually written in a way that can be added to NixOS (or nix-darwin for OS X users) to integrate with the main configuration so then when you're declaring users you can also provide a configuration for each of them.
So it all depends what you want to do, the main configuration.nix is good enough if your machine to run specific service, that's pretty much all you need, you don't care about each user configuration in that scenario, you just create users and start services using them.
If you have a workstation, home-manager while not essential can be used to take care of setting up your individual user settings, stuff like dot-files (although it goes beyond that). The benefit of using home-manager is that most of what you configure in it should be reusable on OS X as well.
If you care about local development, you can use Nix to declare what is needed, for example[1]. This is especially awesome if you have direnv + lorri installed
When you do that you magically will get your CDE (that includes all needed tools, in this case proper python version, you also enter equivalent of virtualenv with all dependencies installed and extra tools) by just entering the directory, if you don't have them installed all you have to do is just call `nix-shell`.
I also can't wait when Flakes[2] get merged. This will standardize setup like this and enable other possibilities.
Chef and from what I heard, since I didn't use it, Puppet are declarative, but since their DSL is really Ruby, it is really easy to introduce imperative code.
Ansible uses YAML, but when I used it few times it felt that you still use it in imperative way.
The saltstack (which also uses YAML) was the closest from that group (never used CFengine, but the author wrote research paper and shown that declarative is the way to go, so I would imagine he would also implement it that way).
If you truly want a declarative approach designated from a ground up, then you should try Nix or NixOS.
I use Ansible for managing my infra, and the only time my playbooks look imperative is when I execute a shell script or similar, which is about 5% of total commands in my playbooks.
One way to test if your playbook is declarative is try to rearrange the states and have them in different order. If the playbook breaks with different order it is imperative.
I would be somewhat surprised to find out Puppet and Chef weren't declarative either. Because setting up a system in an imperative fashion is ripe for trouble. You may as well use bash scripts at that point.
I've used Ansible for close to 10 years for hobby projects. And setting up my development environment. Give me a freshly installed Ubuntu laptop, and I can have my development environment 100% setup with a single command.