One reason I like working at startups is you get to wear many hats. Of course, by "wear many hats" I really mean "suffer occasional periods of extreme stress when things fail and there are no grownups you can go to for help". I like to think of it as Extreme Learning.
  • lemmyng@beehaw.org
    0·
    1 year ago

    Good article for discussion.

    Health checks is one situation where kubernetes really shines. It makes a clear distinction between readiness probes (when the pod is ready to start serving traffic), liveness probes (when the pod should be considered dead), and startup probes (when the pod has finished bootstrapping). Coupled with autoscaling it then becomes acceptable to have a pod stop serving new traffic when it’s too busy, because other pods can be created in a short time to take the extra load.

    Including backend checks in your application depends on its nature. I think the mistake that the article’s author made was not to include the checks, but to have too big of a blast radius when the check fails.

  • lando55@lemmy.world
    0·
    1 year ago

    Essentially, all these gotchas boil down to two principles: don’t mark instances unhealthy prematurely and mark new instances healthy as soon as possible.

    So basically tune your health checks