Hard to disagree. I can usually tell what type of type of experience devs have by the snarky, dismissive responses I've gotten on various internet forums over the last two decades. e.g. "Oh but you would never have this problem if you performed proper code review" - Random_Rockstar_Dev_254
However not all legacy projects are bad to work on. If they're decently developed, then often you'll find that most of the pain is in setting up your local codebase. e.g. sorting out make files, or header file clashes etc. And if you're lucky, some poor bastard has already done the hard work for you.
As an aside, I know tons of experienced "Senior" developers who just suck at their jobs. The problem is they have tons of experience in delivering terrible products based on terrible code and architectural decisions. They just never had anyone to show them any better. And now that they're "Senior", noone can tell them anything. Many devs who work in corporates understand this pain. Shoutouts to my peers who have to "fix" 3000 line stored procs with a 100 line "change control comment saying stuff like "2009-04-03 JS Added TAX_REF_ID". I live your pain.
EDIT: Also, if you happen to work at company that thrives on terrible legacy products, try to drill it into their heads that BAU Support is not part of the solution. Every time I've raised the issue of the mounting tech debt I've gotten the response "Business does not have the apetite to solve these issues. And why would they? That's why we have support teams".
I see that going through these three kinds of projects let me grow as a developer:
1. green field project
2. other people's legacy project
3. your green field project growing into legacy project.
You can learn so much from each of these, but to me the most eye opening experience was our green field project growing into a project with more and more developers.
You could learn so much about others, some were very arrogant, went on constant refactoring mission only to mess up everything. If for some reason, I couldn't check what they did, usually, I had to come in and fix their stuff, but sometimes the only person knowing about the edge case was me, so I just left it "messed up".
Others tried to understand why the system ended up this way, some accepted it, while the best actually improved the system by looking back and recognizing the simplicity hiding in the mess.
My experience, is that this is usually a defensive shell around personal insecurity.
On the outside, it looks the same, but internally, insecure people can be reached (not easy), whereas truly arrogant folks (a lot more rare than you might think) cannot.
My experience is that most difficult people are actually decent folks, that we can enjoy working with, but we need to adjust to them, and they need to adjust to us.
> The Kindle runs stripped down Linux with a React Native frontend along with Java backend applications.
Honest question: what exactly is wrong with this setup?
There's the OS, there are services and user land apps written in java and misc native code, and from the link there's a single component where Lua was used as bridge code.
The GUI is React Native, which is just a framework to simplify the design and implementation of native GUIs.
Well, I have found that finding things in common with people is usually a great segue into improving relationships. Learning more about them, also helps a lot. Find the things they care about. It can be surprising.
I have learned to make the first move, myself. It doesn’t work, if I insist on them doing it. Also, I need to accept their value system. If they truly have values that I can’t deal with, I usually “compartmentalize” things, so I don’t deal with them, in those areas.
I’ve spent my entire adult life, getting along with people that don’t play well with others. I haven’t always been able to get through, but I’ve had more success than failure.
> Also, I need to accept their value system. If they truly have values that I can’t deal with, I usually “compartmentalize” things, so I don’t deal with them, in those areas.
It feels like working with all sorts of people (and sometimes working around them) is an important skill that’s not usually focused on enough, which is how you get people that might be brilliant but also are really hard to work together with.
Not OP but: destigmatize admitting failure (by doing it yourself), normalize admitting ignorance and asking questions (by doing it yourself), find someone they look up to and demonstrate healthy collaboration dynamics with that person, explicitly labeling things which you value, as they happen (e.g. "thanks for saying you're not sure, or I would have thought that you were and it would have made me value the statement differently", "thank you for not taking the feedback on the code personally, the result is a better codebase for everyone"). And do the same with them whenever they imitate any of that, without dwelling on it any further or treating it any differently.
TL;DR: Normalize healthy dynamics by example.
Obviously just my 2c and every case is different etc.
- Share credit and be generous with sharing
- It is better to error in the direction of over sharing credit than under sharing (taking credit for yourself)
- Slow down, say "I don't know" (good to follow up with "but I'll find out")
- Saying "I don't know" is often a better answer than guessing
- Guesses are fine, but should be prefaced and be clear that you speculate BUT will follow up and verify
- Not knowing is the first step to knowing. You've turned an unknown unknown into a known unknown. That's a big step! And often one of the hardest!
- Encourage juniors to speak up and not be afraid of raising issues, even if those senior to them disagree.
- Either the junior provides valuable feedback that was missed (everyone is fallible) or it is a learning experience for the junior and //we invest in juniors//.
- Seniority does not make the dev, the logic/ideas/thinking does.
- Mentor more. I think we often want to send juniors to their corner so we can concentrate on our work, but you have to train those who will later on become your peers.
- There are no graybeards without noobs
- Encourage more collaboration
- Especially when there are complementary skill sets
- Drive culture that recognizes that ones ideas are not a reflection of their value. What I mean is that it is okay to be wrong. The idea is distinct from you, and you are more than just one idea
- We're all wrong, it is just a matter of "how" wrong
- Having a "bad idea" or "wrong solution" doesn't mean /you're/ wrong/bad.
- Recognize that we're all on the same team. Criticism is constructive
- A critique is not a dismissal, it is because we are all on the same team trying to make the best thing. An outside view is often helpful to finding issues. We work as a team, so criticism is debugging as a team, just as you criticize your own work when initially formulating ideas (you reject some ideas, find limits, and formulate plans. Criticism is the collaborative version).
- This means also be careful how you hand out criticism. Ensure it is always constructive. It always contains an explanation and a path forward (as much as possible).
- Competition can be beneficial, but it can easily create tribalism. Fight the natural tribalism because we're all on the same team (helps to have team mixing)
- Make time for learning
- The job always involves learning, no matter how senior you are. If there is only time for "doing" and no "learning" then you will only gain debt and you especially are teaching juniors the wrong lesson.
I would state that the first point can be a bit tricky. I've worked in some environments where one or two people follow the point and no one else does, so this ends up with those people being seen as "non-productive." This is more representative of poor management though and I'd push back against the toxicity to the extent possible.
For managers: the last point is really important. Your coders shouldn't always be sitting in front of their IDE or terminal typing code. They should be spending a fair amount of time reading docs, writing docs, and googling. Writing docs is actually important as it helps communicate ideas, which also serve as reports to YOU. This communication also helps de-abstract the ideas and often leads to finding issues. It is an important step to debugging. (It is often a good idea to have juniors build or append docs when they are onboarding. Dual serves as the intro to the codebase)
One of the best ways for me to learn, is to explain.
That's a very old pattern.
That's one of the reasons I write stuff that no one reads[0]. Writing up a technique forces me to research it, and break it down into its simplest components.
> One of the best ways for me to learn, is to explain.
Kinda The Feynman Method[0] or even Rubber Ducky Debugging[1]. I found that teaching in grad school has immensely helped. Not only do you have to solidify concepts to a much higher degree but you revisit stuff too. I think there is a big difference between the level of expertise needed to be comfortable working with some knowledge vs what is needed to teach that knowledge. The former is much more personal. But when learning any topic there's always things you had to table, that you didn't answer, because you had to make general progress. But if you don't revisit these, then they always remain unknown. Revisiting often brings new insights and thinking that you probably didn't expect. Any time I've revisited a topic, even ones I know quite well and have revisited many times, I often find that there's still more for me to learn. There's infinite depth, so always keep your eye out.
One of the places I feel like I grew the most, and I was lucky to have that early in my career, was freelancing. Every month I'd be ushered into a totally new project/codebase, with virtually no time/budget allocated to "understand" the codebase properly but still needing to deliver value (which was a stark contrast to my previous hobby dev time).
Wow this is a great formulation. I’ve started becoming senior more recently, for most code I’ve become pretty good now at making the tradeoff of how much refactoring to do versus a quick hack based on risk. Sometimes adding in an if statement makes more sense than a full upfront refactor.
The exception… is for any feature I added myself. If I spent a lot of energy releasing v1 with the “perfect” (to me) abstractions, etc. I get paranoid about more junior developers messing it up for new features. Re-reviewing code changes, insisting on all the upfront refactors and rare edge cases handled, etc. Something I need to overcome.
"3. your green field project growing into legacy project."
You do all this upfront design about how it's going to work and then "Oh god there's so many splinters and sharp corners, it's only getting worse and every new team onboarded to the framework needs to have their hands held, and management won't allocate time to address the tech debt."
Ugh, too right. Going through that made me appreciate working on a legacy project later on, because I could improve things rather than watching things get worse.
> I can usually tell what type of type of experience devs have by the snarky, dismissive responses I've gotten on various internet forums over the last two decades.
I see this constantly on HN and have had to learn to not engage with those conversations. It is like some people have never had to run a project under any sort of external constraints like head count restrictions, layoffs, inheriting a system which you didn’t write, or awkward organizational structures.
Some will scream until their ears bleed that there is only one optimal way to do things and if you aren’t doing things that way you are a incompetent chump — totally disregarding the reality that people have to work with.
The real miracle is when you see a teammate who can still perform well under these insane constraints. I have seen it a couple of times in my career. They still find a way to write "good enough" code or write "good enough" docs. It was inspiring.
I would say the same about a sales person who can still manage to have a great year even when the (their) market is cratering. Again, I have seen it a few times, and it always impresses me... watching them pull a rabbit out of the hat!
It feels like you're just finding other ways to describe people who are senior in title only which is far too common. Any senior that uses their title to justify a 'you can't tell me way to do attitude, is a senior in title only.
I think the term is the issue. Senior development means something more intrinsic than it does in other title, like a senior manager. I think what we're attempting to define is something closer to seasoned developer.
> I think what we're attempting to define is something closer to seasoned developer.
I'm fully aware of that, which is why "Senior" is in double-quotes, but experienced (aka "seasoned") is not. My point is that you can be seasoned at delivering bad products. The point about seniority just speaks to tenure at a company. Sure, you can join a company as a "Senior dev", but that's not quite what I'm referring to here. One would think that they would be exposed during the interview process, but alas, we all know that's often not the case.
The title is more or less meaningless these days. That said the other problem is people can't always appreciate the perspective of a more seasoned developer. Some people who are very junior or intermediate think they know better since they don't have the experience. Lots of things in software come down to judgement/intuition and are not black and white. A lot of software development is about working with people and culture, not code.
The unseasoned developers are sometimes very keen on some idea, philosophy or method and are very intolerant of skepticism from someone who knows there is no such thing as perfection and no one way of solving all problems.
But this will never be an issue in the real world
- Also Random_Rockstar_Dev_254
I see this a lot too. Which I think I have a good analogy: judging a program by its output is like judging a math proof by its last line. But math proofs are really hard and annoying because you can divide by x somewhere and then have to contend with the fact that x cannot be 0 or else you have to divide by 0, making your proof invalid. The proof is only valid in its entirety. I see programs in the same way. But this is exactly why Test Driven Development is so naive. You can't just write tests to check your correctness, it only works under the assumption that you can predict all possible classes of data that will be processed and in which way. Currently and in the future. The experience of a senior should certainly make TDD far more effective, but I'd say a graybeard is one who knows the limitations and is able to write code that is efficient, can per-emptively address future tests, and writes in such a way that the code can be easily modified and adapted for the future changes.
If I've learned anything in my history of coding it is that where the program ends up is never where I expect it to. My experience makes me better at predicting that differential, but to be honest, given a big enough project you are never really going to be able to predict the end state. Things change that are out of your control.
I'd also argue that this is why it is so important to write documentation and comments while coding. It'll help you later on and anyone that is onboarding (small cost to document, but many people reap the benefits). I don't want a junior coming in and completing a task fast, but I want them to come in and learn the codebase and how to adapt it for the task. Sure, it isn't as fast, but juniors are an investment anyways. I think we have to be honest that no matter what we do we are either investing or taking debt, both of which compound. But I think the problem is that interest is hard to observe and debt is less observable than the direct cost seen in an investment. Maybe a real Sr Dev is the one who is more aware of these trade-offs?
> As an aside, I know tons of experienced "Senior" developers who just suck at their jobs. The problem is they have tons of experience in delivering terrible products based on terrible code and architectural decisions.
It's a classic tragedy of engineering; you can actually ship with almost anything, and just successfully delivering a product reinforces a feeling that your choices were good, no matter how arbitrary they were in practice.
People can get really confident that what they're doing is tried and true, while it's simply the only approach they've had for all of their career.
> I can usually tell what type of type of experience devs have by the snarky, dismissive responses I've gotten on various internet forums over the last two decades. e.g. "Oh but you would never have this problem if you performed proper code review" - Random_Rockstar_Dev_254
Yes, and the flip side to this is the other kind of senior that you mention in your comment: the one that refuses to believe that things can be done better.
It's hard to find the right balance between wanting to write better code and pragmatically dealing with the realities of a huge, messy legacy codebase with tons of bad decision - you can't fix everything (and certainly not at once), so you have to choose your battles wisely.
The windows kernel is old. The Linux kernel is old. These are both old code bases, but they're not "legacy" at least in terms of how many would phrase the term. A codebase let to rot is legacy. A codebase that is constantly improving itself to be in the best state so that it can adjust to modern programming standards is just a good piece of software.
That all said, it's all subjective, blah blah the end.
Maturity and humility are key factors in being a senior developer. Those who lack humility will often cry about how its not their code that's messed up, delaying actual bugfixes and wasting developer hours, or worse producing a toxic work culture.
You can learn from anything. A "legacy" product is one that works, but is generally hard to maintain.
"Why is it hard to maintain?" is a question every architecture person should be able to answer...especially after digging into some large, old piece of software.
As an embedded engineer in a new position I always volunteer to take up an old crusty legacy product that nobody else wants to touch. It's a quick education into how the company approaches projects and all the ghosts they had to fight along the way. All the business logic is there, and sometimes it's even documented.
And once you have a semblance of knowledge about how the thing works and how to make fixes, you've built a small moat around your career there.
This happens to me all the time, but mostly with work code.
I'm under pressure, I have to deliver, things get rewritten many times, edge cases are found and fixed in production after they slipped through multiple qas but need to be fixed instantly or we lose money.
It happens.
But personal code? I am almost always proud of my work. It shows in general I was not under pressure and enjoying building it.
Even my code from the first year of programming, it shows it was made to make a point in writing a nice solution.
I may cringe at times, but still proud.
Context does matter, and code quality is more often than not a byproduct of our emotional state rather than skills.
I'm doing my best to prevent myself from being on thin ice with my future-me.
I posted recently about how I divorced not too long ago. I'm in an apartment now (for the first time in 16 years) while I stack up cash to build a new house.
I always think I'm not making any progress but then I see my next door neighbors ordering DoorDash 15+ times per week, and I realize that I'm doing pretty damn good for myself.
Or I walk by the mirror and think about how I've neglected my lifting for the past 6 months while I finished up this SWE degree. But I'll see those same neighbors struggle to walk up the stairs and I again realize that I'm making some awesome choices compared to those two folks.
I’ve found one of the best things I can do to help my jr devs feel more confident (and be willing to ask for help) is highlight my screwups and “basic” things I’ve just learned. It make me seem more approachable and less like everything I say is a pronouncement from on high, it makes them more willing to admit to their own mistakes, it clarifies that we’re all learning stuff all the time and it helps spread knowledge about things that folks might otherwise be too afraid to ask about for fear of looking silly.
A benefit of being a sr dev is getting the benefit of the doubt on mistakes, so I might as well use that to my advantage and let everyone learn from them, not just me.
I 100% agree with you, and have used the same technique. Showing juniors that you are also human and make mistakes allows them to be less anxious about learning and trying new things. It also helps to keep them from trying to hide their mistakes which can quickly lead to larger system issues.
Of course, you have to have a culture where making a mistake doesn't immediately lead to being let go or publicly reprimanded. I have definitely learned quite a few things about people and code by helping juniors work through bugs that may not have been so obvious (including occasionally finding bugs in 3rd party libraries).
I used to suffer from imposter syndrome significantly. It helped to see the mistakes others made which anyone could have made. It also helped to have my own good ideas and talent explicitly acknowledged. Learning to not take criticism personally and own your own mistakes also helps to rob imposter syndrome of its power because it demonstrates to others that you can be forthright and self aware, and that helps build trust.
Sure, but if there's anything I've learned from working on legacy software, it's that I don't know why the code was written the way it was. I'm not the only smart person to work on a codebase, and it's more common that another smart person did what they did for a reason, than that I'm a genius who knows better than they did.
Sometimes there were reasons, but they were not necessarily good ones. Like these:
- time pressure, leading to just making it work and not caring about edge cases, which were bolted on later. Or maybe never at all. And naturally, there is no useful testsuite.
- the prototype ended up in production
- a refactoring happened, but there was no follow-up into other areas of the code base
- features changed, but the code is still influenced by the necessities of the old way it worked. Similarly, for changing technical circumstances
- code was copy-pasted or was worked on by too many people
- code was herded too long by the same person and now, to put it nicely, bears traces of their bespoke coding style
Chesterton's Fence is still very relevant, therefore I'd still err on the side of writing a lot of high-level tests before touching anything.
Kind of light fighting against the human tendency [0] to blame the failings of others on their core character rather than the situations they are in, and vice-versa for oneself.
I don’t think there was any dig implied. And I will second that it will add to self esteem. Specially during discussions if you have more understanding about the code you will have both better contribution and take away
Sure, when you succeed in understanding someone else's legacy codebase, it builds self esteem. When you fail attempting to do that, it does the opposite.
I have inherited projects where I just could never understand what they were thinking well enough to make the service good (not running out of memory and crashing). It was very humbling; I still sometimes wake up in the middle of the night with things I wish I'd tried, but the assumptions of how it all worked was just scattered throughout a bunch of files and the assumptions of everything was just not how I'd do it.
Self-esteem that easily turns into hubris though. I think the real seniority shows when you are able to work on a legacy codebase full of the shittiest code and not have the slightest desire to rewrite it all.
I think that value comes from code that solves tough or complex problems, which can be humbling while improving your understanding. Feels good when you get a grasp of what’s going on and can understand that there may not be a simpler approach, just a different one.
There’s also legacy code that is not great and is not solving a complex problem but is just a product of the conditions at the time. It’s easy to look at that and assume you can do better given the same constraints, and this is neither educational nor esteem-building. Just an ego massage.
> Engineering isn't about working on the most interesting problems. It's about getting stuff done and management happy
Truth is harsh, however this seems to be 100% accurate for nearly all cases of employment. Rarely do you get to focus on simply interesting problems and good engineering as a primary concern
Boredom is in the mind, not the task. Things aren't boring, people are. An important type of intelligence is the capacity to find what's interesting about a task that others lack the imagination to see. One needs to be able to create their own interesting solutions rather than expecting them to be handed down on a plate.
> find what's interesting about a task that others lack the imagination to see.
word of warning from an old guy, don't create problems for yourself when you don't have to. Turning something boring into something interesting can have painful consequences down the road.
Boredom is lacking stimulation. Even the most cutting edge task can grow boring if you need to plumb with it dozens of times over a year. Just because you can find what's unique doesn't mean it stimulates you. That's the exact issue with why neuro divergent individuals are demonize: they don't take as much interest with people as "normal people" would.
>One needs to be able to create their own interesting solutions rather than expecting them to be handed down on a plate.
> You should tell that to every old manager I had.
They sound like boreful people. If you find you have become boreful, there's a good chance you may be experiencing burnout or depression, which are nasty diseases, but still ones that afflict the subject, not the object. Nothing is boring but for the person who perceives it that way.
They have a job and family at the end of the day too. Not everyone has the power nor passion to try and and move a billion dollar corporation to care for people over profit.
>If you find you have become boreful, there's a good chance you may be experiencing burnout or depression,
I haven't has a full time job in over 18 months, so I hope I'm not burnt out.
>Nothing is boring but for the person who perceives it that way.
That's like saying nothing is ugly except for what people think is ugly. Boredom is personal and shaped by each person's experience, so I don't think framing it as a person for not trying hard enough is going to go too far in practice (nor is it productive to judge people based on how they prioritize their lives). Of course some things will be boring to one person and a life passion for another.We don't have enough time nor energy to try and find appreciation for every object and concept on Earth.
You gotta go into R&D if you want to focus on the fun stuff without the annoying plumbing. But such positions require an entirely different pipeline from getting a SWE position out of college.
A good way to secure your position is to be the go-to expert for a product with many years of life ahead of it.
Fixing stuff on a legacy product may make management happy but if that product is discontinued next year then you haven't accrued technical expertise valuable to the company (but you may have built a reputation as a fixer and quick learner).
So, as usual, it is a balancing act.
Edit: this is my perspective from the embedded world. It probably applies generally, though.
When times get tight the new projects get shitcanned and the 10 year-old cash cow design gets the promised new features.
One crusty project I worked on was a legacy control board for a piece of restaurant equipment. The customer, the company that built the actual machine, had been building this product for 40 years. It had been through two PCB redesigns and two different microcontrollers, but the logic was tried and true and had to survive. A port of the project from 6800 assembly to C had completely gone off the rails and the contractor was dumped. All it took was a 20-opcode fix to a routine that the contractor just couldn't grok.
I wouldn't say that's the conclusion. If there's only one true thing about work is: management doesn't care about you. They can fire you for any reason, and thinking that by working on stuff nobody else wants to work on you are "safe", it's an illusion.
If any the conclusion is: work on what you want, life is short.
Engineers have a special place in society like doctors and lawyers. Working with management is part of the job, but engineers have a professional ethical obligation to say no if they are asked to something against the public good.
The split there isn’t in favor of doing stuff that’s fun and novel though; actually, the engineer should usually pick a boring proven solution if the public has a high stake in the outcome.
> Engineering isn't about working on the most interesting problems. It's about getting stuff done and management happy.
That's a perfectly reasonable thing to want out of engineering for yourself. I wouldn't state it as an absolute truth for all people though.
Personally, I'd like to be working on something that extends the state-of-the-art a little, even if only by a tiny fraction. It can be one for the other disciplines involved - it doesn't have to be the software I'm writing that is responsible for that (and it usually isn't), but that's what I derive satisfaction from.
Do you think that software becomes poorly understood and maintained because the company treats it as a prestigious job and rewards people for working on it?
This is how I got laid off. Working on legacy software, sole person on the team, eventually management decided that it could be replaced by AI or some such pixie dust.
Legacy software with a single dev can be on the fast track to getting shut down. If it was still a business priority, they'd be throwing more resources at it.
That's a very narrow definition of engineering. And while it's not wrong, it's absolutely more of a "management" POV. Like sure, for management, engineering is mostly about what you said, but that's it.
I find improving established code of far more interest and reward than greenfield. Coming to understand what a person was thinking while building something is fascinating. My updates are tangible life improvements to real human beings. That interests me, at least.
Greenfield, I have the incredibly stressful job of making all the major architectural mistakes people will complain about for years to come. I'm not interested in that. There's also a much higher chance with greenfield that the product will just fail and no one will enjoy your work.
I should say not prevent it. But minimize it in the maximum possible way.
I am sitting on a gold mine. But I can’t prove it. That’s the problem with this stuff. The industry moves horizontally with each design such that nobody knows if say the current design trend was better than the last. We go better or we go worse and nobody knows if it actually was.
Maybe I am sitting on a gold mine. The problem is you’d never know about it. You’d never believe it. Because a proof isn’t possible.
So because it’s not provable you have no clue about any design whether one is better or one is worse and thus you disbelieve everything.
The overall question is how do you create a design or pattern that has maximum adaptability to anticipate any possible requirement change with minimal rewrites? Where you only swap in and swap out modules without changing structure?
I think I found the best possible way. But you won’t believe me.
I'm not the OP, but I'll say that I don't believe you, because you're telling me you have The Perfect Solution, without providing even a hint of what that solution is. If I was skeptical at first, I'm now certain you're selling smoke & mirrors until you provide any evidence at all you actually have an idea here.
I literally said I don’t have a perfect solution and that you wouldn’t believe me. What I have is the best possible solution based on available solutions that we know of. I found the best way.
I understand what you are saying. Obviously the type of development you do is a factor, as well as the way you and/or your team does development. I think others are looking for the one true solution, and you are saying that you have found the solution that works for you, given the way you develop and the constraints you deal with. At least that was my understanding of it. If you are doing embedded development, being thrown into a large front-end development project isn't going to use the same solution.
I have a solution for custom content management systems that usually makes it fairly easy for me to incorporate customer requests without needing to toss out my existing solution and start over. Almost a type of framework built over an existing open source CMS product. I maintain about 90 websites as the only backend developer, and have spent about 12 years now working through various versions of the software to custom tailor a solution that works for nearly all situations I have come across. I don't do front-end development any longer, but I was involved heavily at the beginning stages to make sure both the front-end and backend were flexible and broken apart into modules.
No im saying i have a general solution for everyone. The general part only applies to the part where are software projects are similar. The part where they are dissimilar it can’t apply. It’s impossible to generalize over distinctly different things but I found a solution that generalizes over what’s similar in the overwhelming majority of projects.
If you find a templates solution works for you then that is not a general solution. It’s better for you but you are hardening and becoming opinionated on certain details and that works for what you’re working on. My solution is a lower level pattern that is more general and less opinionated. For what you do it likely is only effective if you are required to drastically change an existing project while maintaining as much existing code as possible. The solution prevents a complete rewrite and allows you to modularly pull out what you don’t need and replace with the things you do need.
If you don’t need 90 percent of your project this pattern won’t prevent that. What this pattern allows is for you to actually pull out that 90 percent. Many projects that only need you to pull out say 30 to 40 percent of it actually end up with a full rewrite because the modules aren’t decoupled enough.
Apologies for the misunderstanding. Sounds interesting, but saying you can't prove it, and nobody will believe it is a tough sell. I guess I find it hard to believe like others, but you are talking in very abstract terms, and not being able to (or refusing to) give any kind of details sounds like you are trolling or don't realize that your solution does only work for what you work on. I'd say I could be proven wrong, but it seems you don't wish to do anything other than generalize. Good luck to you if you've actually found what you describe.
No need to apologize. I assure you I'm not trolling. Yeah I am talking in abstract terms because I don't really want to get into a debate on the topic. The reason is because any design style in this area is basically unprovable and leads to endless debates. There's no point.
I do have something that explains my pattern and serves as support for it. But it's no proof because such a thing isn't really provable. Anyway it's easier to let AI explain it. In the following conversation I basically ask chatGPT about this topic with very general questions while manipulating it as little as possible.
chatGPT basically arrives at the same pattern I discovered. Where I cheated is when I started giving my own opinions on the several patterns chatGPT suggests and I hinted the LLM at the direction I wanted to go. Everything else is a very general question.
That being said, there are clearly other sets of general questions that can very well lead chatGPT to form alternate conclusions, but this one aligns with what I and many others have discovered.
Those are pretty well known FP oriented guidelines that many developers here on HN are familiar with. I don't think that your fear of not being believed is warranted.
HN is mostly not hardcore fp programmers. There’s a good amount of fp programmers but they are not the majority. Plenty of go and c++ lovers here and people who don’t understand the guidelines above.
The point is I don’t want to argue about it with someone who doesn’t get it.
I pulled this strategy at a new company and found that it opened up a million opportunities I could take my pick of. I basically got to shape my own role going forward.
Why? Because I very quickly made a name for myself as someone who learns quickly and then gets stuff done even in contexts where everyone else struggles. Solving problems that have long been considered intractable is a good way to show your value.
This is 100% true. Those problems you are solving probably don’t look all that important, but, in reality, they have been blocking sales, renewals, account expansions, or even access to markets for a long time. The business side of the house has probably given up or occasionally tries to wag the dog to get it fixed.
> Haven’t you also often made a moat between yourself and doing literally anything else more interesting at the company
Nope, assuming that code works and handles UTF8 (sorry, but there is a point where it is too far gone to save). You’ve made yourself essential. You know the business better. You are strategic. And often, you become the one person who knows the legacy code base. And you will quickly find yourself a part of super important decisions. You’ll be called out regularly for fixing things that have gone unfixed for years. You’ll also drive value by helping the company avoid spending vast amounts of money re-implementing functioning code. On top of that, new product is often a mix of new and old, so you’ll quickly find yourself in the middle of new product teams.
These insights are essential for playing the long game; it may sound distasteful for people early in their careers. That’s fine, but when you’re running out the clock to retirement, can’t go wrong.
The most interesting work is going to be 10% interesting and 90% not interesting. In most cases it will be less interesting than that.
Prioritizing "interesting" work means you'll always be exploited by the companies that can sell you on less pay, less benefits, less time off, less respect in exchange for some pittance of perceived novelty. Don't fall for this trap.
For what it's worth, I find it much easier for my ADHD to debug and improve existing code than to greenfield - it's actually interesting. Debugging is like a detective's work with small dopamine hit at each step of unraveling the situation and a huge reward once you finally grasp it. It's best to do on someone else's code, so you don't get too frustrated with your past self ;) Greenfielding provides some rewards too, but once I'd greenfielded anything remotely similar already there's a lot of forcing myself to do it around the beginning and end, where there's plenty of unrewarding work to do. For greenfielding to be fun and easy, it needs to be a completely new area that I had no previous experience in, but that doesn't tend to end up with excellent results.
On the other hand, the infrastructure variant of this is the exact opposite. Trying to figure out if something specific is in use and who uses it or if someone just clicked around in your cloud console and forgot about it is the worst. There's only so many ways I can be inventive in building proxies to measure usage.
We sometimes get incidents that things someone remembers should exist do not work, only to find out that that product was retired several years ago (but someone forgot to delete some DNS records, so there's an error message from... someone's server!). But only because we asked half the company on Teams.
... I literally had a multi month ticket open to track down an expense for something I either never used or only used once which ended up being a load balancer or some crap
As a senior engineer your job is to take on the unglamorous hard problems, both technical and organisational, that are holding the team back. Doing this, and being seen to do it, early-on in your time with an employer will be somewhere between appreciated and expected.
If you do this you should also be good at story telling. Not working on the new things, but making sure the core business running, is often times looked down upon in my experience. Most leaders will see the greenfield project move fast and wonder why the established and "complete" software moves slower.
I found that working on legacy projects is often the most interesting.
There is the challenge of understating the original code base, which may look messy, but is full of history, you see the architectural mistakes and their consequences, how they managed to fix obscure bugs, tackle unexpected use cases, different code styles,... Then they ask you to implement a feature or fix a bug, you have to find what can be touched and what can't, what is a bug and what is a feature, see how much time you have, and decide on a strategy. This is production code and people are using it right now, and they are efficient with it, you have to help them and not force them to relearn everything for no good reason.
And there is technical debt management, your goal is to make the code better or at least not worse, but without introducing breaking changes and under time constraints. You have to manage your off time to make improvements that will help you in the future (chose wisely) as the next truckload of tickets arrive. It includes having a good relationship with your managers as no one wants you overworked, as it would just cause technical debt to accumulate with no time to repay it.
And good luck getting an AI to do that, if you want job security in the age of LLMs, that's your change. It doesn't mean you can't use AI, but you certainly won't be replaced by it.
Nope. If you're willing to take on the shitwork without a lot of handholding and show real progress (which actually impact earnings) you'll have a really good stab at new opportunities.
Small, but not insurmountable. The time always comes when a new leader and their lieutenants come into the org and prioritize new features over maintenance, at which point your moat is null and void and you’re on borrowed time. I’ve yet to find a way to build a moat that withstands that level of hostility without actively harming the organization in the process.
I’d love to hear how others deal with this inevitability.
When you're on a reasonably large legacy project that's a pipe dream.
One of the old specialty C ERPs I've worked on has literally buried more than 3 (the ones I know of) "great Ruby rewrites". There are just _so many_ features, that the newcomers always ended up on the chopping block after 5-10 years and before they could achieve real customer traction. The old project just ended up getting some more interface fixes instead (aka even more features to catch up to) and to this day the compiler for the core hasn't even been upgraded to a version that can run on anything newer than XP SP1 (the improvements in SP2 break it).
Now they're doing another modernization project with some web services and stuff. We're happy to take their money for those as well. I'd be quite shocked if those do anything other than enlarge the moat of the core.
Yep. The only possible way to rewrite such projects is to do it gradually, piece by piece. It is also important that the refresh happens from within and is led by those who understand the legacy codebase and, especially, the business requirements it fulfills. No way can an external team do something like that. Being given project description "it must do everything the legacy project does, plus these new features" doesn't help either. If you get a ticket like that - run.
I just spent 5 years of my life building such a system. Then we were asked to do it again (a refresh of the refresh, if you will) and everyone marvels that suddenly we can do everything both fast and correct.
Well, we just spend 5 years training the team on how to build the old refresh, and now the new refresh can be done properly.
> The time always comes when a new leader and their lieutenants come into the org and prioritize new features over maintenance, at which point your moat is null and void and you’re on borrowed time.
The "on borrowed time" is an odd conclusion.
When someone takes over, it's typically a priority change ("We want everyone working on new projects, just freeze this old stuff") rather than some kind revenge path against anyone who worked successfully on the old stuff when it was a priority.
And a good engineer who is able to handle legacy code will typically be even better when working with greenfield code, because the latter is easier, all else equal. So unless you are refusing the priority change, why would you be on borrowed time?
It does happen that the new leader ends up saying out loud "everything we have is terrible," which is easily taken as "everyone who worked on it is terrible." And to be fair, sometimes that's true. And sometimes it's not true but the leader wants to bring over their crew of loyalists and needs to make room somehow.
Long-term ownership and careerist jockeying don't go well together.
My recent experience is that a lot of the system was not as good as it could have been because it's a hard problem. The company got bought out and all the new bosses come in and say all the existing code and way things are done is crap. Then, a year later, with everyone sacked and new people brought in, once again, these guys say the same thing and promise they are the top guns to fix it all up. I just checked out on the third round.
Ah, that's definitely true. But that feels like an orthogonal concern to you being an existing dev working on legacy stuff vs greenfield stuff... unless the new boss likes the greenfield stuff for some reason. What you are talking about is more of an ever-present possibility of "new management fallout" in almost any field.
if you've built up respect at the company and speak your mind regarding the maintenance vulnerability, the new crew should pay attention. if they don't, then you're screwed anyway, but so is the company.
source: endured this while working at a startup that was bought out by SFDC. Kept my principles, but lost the job. /shrug
Yup, you and Shermantanktop get what I was asking perfectly. Seeking confirmation I hadn’t overlooked anything that I could’ve done better, but you’re both dead-on.
The irony is that for those of us who enjoy maintaining, improving, and eventually replacing legacy stuff (code, systems, infrastructure, etc), what we really want is often just stability in career - hence the moat. It means it’s cheaper to keep us around for the long haul than rehiring us again at current market rates every few years, especially if we’re making ends meet with a high quality of life. But that’d mean focusing on the long term, which I don’t see many companies doing in general.
This can go wrong, as I found. Management and Tech leadership are focussed on the new projects with latest technology - they pay the biggest bucks for these and no one really cares about the devs maintaining the old stuff - even if important. The second problem is looking for new roles you resume is filled with old tech rather than new stuff that is in demand.
I’ve found it to be the opposite but it’s not really either/or.
Over here in the UK, if I constantly chase the new hotness I will have plenty of new startups to choose from but I’ll almost always be low-balled on pay with some practically meaningless equity package.
Gaining more in-depth experience opens up the better paying positions further up the ladder in more established companies.
Don’t want to keep your head in the sand for years but at the same time, there are a lot of skills that can’t be learned if you stay exclusively at the bleeding edge.
The moat is real. A shiny new project can always be cancelled, but withdrawing support for a suitably large legacy application that doesn't have a replacement is harder.
Also, some companies and teams do versions of this, officially or unofficially.
One pre-FAANG tech company I worked had a policy (known among the local schools) that each new hire has to "pay your dues" on grunt work, before you get to do fun/glamorous work. That kinda worked for them.
A problem they had was that they were doing desktop development in C and C++, which, among other purposes, had to ship software bundled with new brand-name PCs (so not necessarily able to bugfix over Web later). The real problem was that very few new grads are minimally competent at production-grade C or C++. But, in early Web years, the company couldn't hire enough experienced software engineers to relocate to a college town. So they had to try to train and retain local grads of a top CS program. That might've been part of the thinking behind "pay your dues"... before you get to make rookie mistakes with greenfield.
Similar here. First thing I do when joining a new project is to fix bugs in a legacy product. Forces me to read through the code, follow the data and execution. Nice thing about bugs is also they (mostly) have defined behavior.
It's all fun and games until eye̶s̶o̶m̶e̶o̶n̶e̶ ̶l̶o̶s̶e̶s̶ ̶a̶n̶ ̶e̶y̶e̶ management demands some new feature and refuses to understand why it would take so long to implement it on such a stable and well-established p̶i̶e̶c̶e̶ ̶o̶f̶ ̶o̶l̶d̶ ̶c̶r̶a̶p̶ project.
Agree. Also, not be judgemental about the state of things as in "which moron did this". Approach it from a point of understanding and respect rather than criticism. There are multiple reasons why the code base and tooling around it are in the crusty old state. They are usually decisions that engineers or middle management had no control over.
I've been with my company for a while now. Too often I find someone come in with the "this is trash and all of you are idiots" mentality. I've typically not been keen to help such folks. They usually soon ask for rescue from the very same dumpster fire they created.
... just don't make the mistake of documenting too well :-)
Developers need to realize that AI means that a lot of coddling our profession used to get is going away. Employment has always been a tenuous relationship with a fundamentally sociopathic entity, but things will be getting a lot worse in the future.
> Experience is what you get when you don’t have any.
The only better experience than working on a legacy codebase is working on a greenfield project long enough to watch it become legacy and see the good and bad consequences of past decisions.
And the only better experience than "working on a greenfield project long enough to watch it become legacy and see the good and bad consequences of past decisions" is working on a second greenfield project long enough to see that drastically overcompensating for all the bad things from the first one is not the right solution either :)
And the only better experience than that is killing a new greenfield project before it gets to production because the old software was good enough and the problems were organizational in how the software was being used.
I wish this happened tbh. I've seen one where the greenfield (Scala, AWS, etc) is still living alongside the old good enough software they went back to (C# / .NET) ten years on.
I thought it was pretty funny how we had this large project that was supposed to replace a legacy system (that was mostly a bad hack that got pushed to production).
But when it was finished it failed to meet the basic requirements for the only customer that used the system.
> how much they disparage the previous team's work.
And many fail to discern between "disparage" and "critique" or even "Question in order to learn"
One of the greatest failings I've seen in leadership in our time is the idea that in order to make a critique one must come with a solution in hand. As a leader I want to know the things that are going wrong as soon as they're seen, not to require someone to go through the heavy lifting of a solution before they say a word. Now, of course, there's a difference between bitter unhelpful cynicism, and simply identifying a gap between the current state and optimality.
> And many fail to discern between "disparage" and "critique" or even "Question in order to learn"
I think I’ve had the conversation with new to my organization devs a few hundred times: “Look... saying code is crap or stupid is telling others you’ve given up on learning. How about asking why it is the way it is?”
> greatest failings I've seen in leadership in our time is the idea that in order to make a critique one must come with a solution in hand.
The pattern works at very high levels in an org chart, but with developers and those that manage them it breaks the whole concept of problem solving. You have to be able to identify and understand problems before you can come up with a solution... and usually, with software, the solution is developer hours.
I always pick apart previous teams' work.. it's how I learn. I question most every decision because I'm curious why they made those decisions. And it lets me think about how I'd do it better. And yes, I know that many poor decisions are not necessarily the developer's fault. It could be bad specs, lack of time, etc.
In most cases, "better" means different things in different contexts. (Customer-driven vs performance-driven, for example) Of course, this isn't news for most of us. Where I think a lot of us fall short is assuming that definition has changed since the code was written.
This harkens back to Chesterton’s Fence. It’s always worthwhile to interrogate why things were done the way they were, especially when first coming onto a project. Knowing the why of a decision is essential to understanding if and how it should be changed. Especially if the reason is “this is what we had the time and knowledge to do at the time.”
> It’s always worthwhile to interrogate why things were done the way they were
It really isn't. A lot of the time you end up spending a lot of effort to understand something that was dumb to start with and has been dumb ever since. Something like the bullshit asymmetry principle applies - any idiot can take 5 minutes to write a line of code that will baffle a team of experts for hours. (I've done so often enough myself).
The people who have those answers have long gone. The only person left is a project manager who tells you it's up to you to figure it out. After you make a change in production they will come to you with questions after a few months, just when you assumed things must have gone well
Oh man this one hits home. I don’t do much coding anymore but my general advice to folks I lead is you’re never going to be happy with how you did things and just make sure it scales and is well tested.
Edit: oh and how could I forget as simple and readable as possible
At this point I just don't think total rewrites from scratch are a good idea, full stop. I've never seen a rewrite from scratch that didn't lose most of the learned solutions from the previous attempt, repeat most of the same mistakes and have to re-discover the solutions, and utterly fail to even attempt a passable improvement on a model of the core complexity of the problem being solved.
I'm not granting a "rewrite from scratch in Rust" exception even though that's in vogue right now. I'm not saying don't rewrite it in Rust, I'm saying don't rewrite it from scratch. It's harder to write new features in Rust while maintaining the old C code, but it's the right way to do it.
I always tend to say: "Everybody learns better from their own mistakes, but if you are empathic you can also learn from other people's mistakes".
The latter is less costly and only requires you "only" to open your eyes and look at projects that are in an ugly state the right way. Yet surprisingly few people are capable of looking at someone elses fucked up project and not going all like: "Hah! Idiots! I would never have made that decision".
Maybe however that crusty piece of code used a framework that — back in the day — was the hottest, trendiest piece of technology out there and you are currently in the process of committing similar sins, and you won't know it till it is too late.
For me adminstration of Linux servers has been an invaluable source of inspiration. You are directly and 100 percent exposed to the effects of software aging in a changing environment. And you directly wittness which software ages like fine wine and which ages more like milk.
cURL is my personal go-to when discussing things like this. The Unix implementation is old, stable, functional, and has changed elegantly with the times.
This is literally the approach recommended in Team Topologies. The same team should own vN and vN+1, so that they both have to operate their own design and have the opportunity not to make the same mistakes again. It should be the default.
It’s okay to enjoy work and focus on your craft for two years rather than jump for more money. If you like your employer and colleagues and are growing as an engineer isn’t that better in the long term? I think jumping around can risk creating an engineer who leaves a place worse than when they started.
> If you like your employer and colleagues and are growing as an engineer isn’t that better in the long term?
No, growing as an engineer just means the reality of software development will make you more miserable.
> I think jumping around can risk creating an engineer who leaves a place worse than when they started.
Of course it does, but companies have made the choice to pay more for that than for someone who stays and makes the place better, we should give them what they want.
> It’s okay to enjoy work and focus on your craft for two years rather than jump for more money.
These aren't mutually exclusive.
> If you like your employer and colleagues and are growing as an engineer isn’t that better in the long term?
Better than liking your employer and colleagues and growing as an engineer and also having more money? No.
This isn't a field with low demand for talent. You can have all these things--don't settle for less.
> I think jumping around can risk creating an engineer who leaves a place worse than when they started.
That's a risk that companies should be willing to pay to avoid.
But a lot of companies aren't willing to given their engineers adequate pay increases each year, and it's not engineers' responsibility to accept less money than they are worth. If you want more experienced engineers, you have to pay for more experienced engineers. That's just basic free market economics: if you don't like that you don't like capitalism.
Some companies understand this and do better, but these are the exceptions not the rule. If you find such an exception, stick with them!
I fundamentally disagree with the narrative that engineers are supposed to be passionate about their craft and learning and interest and not care about the money. That's propaganda spread by corporations to get us to accept less pay, and if you believe it you'll be exploited. And as it turns out, the places that pay the best are also usually the best places to be passionate about your craft, so there's no real conflict. The places that pretend to pay you in learning and passion instead of money generally don't deliver on the learning and passion either.
In particular, I appreciate that you assign value to the consequences and not the decision itself. Anytime junior engineers on my team would complain about "shitty code" I'd assure them that someone would be complaining about their code in a few years.
Having the context or, better yet, responsibility for the past decisions is great for developing a pragmatic approach to software design AND empathy for other software engineers.
Came here to say this. Ive had to work on slop, and it never stops being slop unless the stars align and you can do a bunch of work which is short term unprofitable. What people really should do is work on the same project for five years. Watch the interest wax and wane and more importantly watch the requirements change. Then you'll stop looking at things in black and white like "good project bad project"
You're also not a senior engineer until you've been forced to adopt the coding style and practices of that legacy project. A lot of younger developers like to come into a new project and slap on the latest frameworks or coding practices that they've picked up recently, without acknowledging that breaking the standards of the project lead to higher mental load to maintain that code over the long term.
It isn’t just younger developers doing this, nor even inexperienced ones. I’ve seen a ton of experienced people do this to themselves and their teams, mostly it seems to be able to pad their resume, but some honestly believe newest and shiniest is best.
If someone has only used a handheld wood sander, they will be ecstatic when someone shows them a powered belt sander.
But someone who has used a palm sander, orbital sander, belt sander, disc sander, and so on will be able to see a new type of sander in the context of existing tools and know what’s right for what job.
Your average junior will be in the first scenario. However, it’s possible to be experienced yet still be stuck in the first scenario.
The way i've seen it is Junior/Senior as a label of wisdom/engineering acumen and experience (not just a title) is not directly correlated to years of experience. Some people just have the same year of experience/level repeated 3-5 years. Some "Junior" with 1 YoE can come away with learnings some seniors never get in their careers. Not wholly their own fault, but it's a fact.
I cannot grok this. Are you saying that “juniors with tons of experience are very common?”
Do you mean that there are seniors with a lack of experience? I believe yes I was also making that same point.
Ageism in the industry also encourages seniors to appear to be plugged in with the newest shiniest thing, so we kind of do it to ourselves with our hiring practices.
“Juniors with tons of experience are common” means that people with many years of experience who are nonetheless at the level of a junior engineer are common.
This phrase is divorcing seniority from talent, albeit perhaps not in the way experienced engineers might prefer.
> A lot of younger developers like to come into a new project and slap on the latest frameworks or coding practices that they've picked up recently, without acknowledging that breaking the standards of the project lead to higher mental load to maintain that code over the long term.
Right. At my new job, the framework-level choices come from people like 4 levels above me. Problematic for a lot of reasons, but juniors slapping frameworks onto things is not one of them.
I’m actually kind of struggling to imagine a scenario where a junior developer has the agency to slap entire frameworks onto existing systems. Maybe a late stage startup? Coding practices, definitely.
We are currently going under modernisation converting Perl to Python. It's what it is, but I've encountered developers coming along and wanting to throw bulky frameworks at when all you require is to process the output result of a vendor issued binary.
There have been novel ideas but just unpractical for the causes.
It's madness getting through to them when all you require is a replica of the current system and not a rewritten feature set not related to the scripts at hand.
Coding practices are another tool with trade-offs involved in their use. Mental load is sometimes another word for learning. So I like to hear it from juniors because whatever is ultimately decided, it's an opportunity for them to have a real discussion around trade-offs and the benefits versus the costs of a particular approach which is invaluable for them.
> A lot of younger developers like to come into a new project and slap on the latest frameworks or coding practices that they've picked up recently, without acknowledging that breaking the standards of the project lead to higher mental load to maintain that code over the long term.
Using the latest framework improves their future job prospects. Whatever company they work for will lay them off in a split second and won't provide anywhere near the comp increases that a new job would. So why should they care about the long term of a project?
Companies have no loyalty to employees which means employees have no loyalty to companies.
> So why should they care about the long term of a project?
Because your future networking potential—and therefore future job prospects—depends on being liked by your peers, who may not choose the same moment of departure as you would. Follow standards and work towards long-term maintainability not because you see yourself at the same company in 2 years but because your coworkers might see themselves there and you want them to like you and feel good about recommending you to their network.
The co-workers that matter the most for networking are the ones playing the same game. They will move to new companies more quickly and try to rise up the ranks more quickly. They will know how to write a good referral because they want to maximize their chances of referral bonuses. They will have the ear of the EMs and Directors because they know that maximizes their own career options.
Saying you're right about which referrals matter (which I don't accept, but I'll play along): you're presuming that not only are these people playing the same game but they're willing to help along someone else who they know has no interests but their own at heart.
Why?
Why would they want to help you along? They burn capital referring you, they burn even more when you make a mess of things. When you leave a burning husk behind everywhere you go your own network of people who trust you is minimal, so you're not an asset to have around them. Why would they want to help you instead of finding people to con who can actually be an asset to them? In this entirely self-centered world you've put together, what's in it for them?
> you're presuming that not only are these people playing the same game but they're willing to help along someone else who they know has no interests but their own at heart.
Referring people for no other reason than that you know them works great, since it will implicitly happen in reverse too. See e.g. the Freemasons.
> They burn capital referring you? They burn capital referring you, they burn even more when you make a mess of things.
They burn capital if hiring you seems like a failure, they win if it seems like a success. That means your incentives are aligned. No-one gets dinged for hiring someone who implemented a couple of buzzwords, left after a couple of years for a higher position elsewhere (or got promoted into management), and then the project they were on failed a few years further down the line for unrelated reasons. Quite the opposite.
Personal pride? I flat out don't respect a developer who refuses to do anything right because of nebulous hatred of any and all employers. No they don't care about you, no that isn't a good reason to go full apathetic.
It's not hatred. It's self-interest. The very heart of capitalism. A rational response to incentive structures.
You're the one sacrificing your own self-interest for pride and are upset that others refuse to do so.
edit: And I get it, I don't optimize my own career like this, focusing more on what I enjoy to work on at the moment. But I don't begrudge people who do optimize nor do I claim to be better than them. If anything they are acting in a much more rational and organized way to their goals than I am.
"More rational" is subjective. There are values and incentives, other than money and specific kinds of self-interest, that can bring satisfaction and joy.
And why should someone be respected if they are intentionally making others lives more difficult like that. That's just hypocritical.
There's a tradeoff here too. If your codebase was started in Java 6, I dearly hope you won't continue to write code in that style anymore for new code.
Not to gatekeep, but I question whether this is even a legacy project, or just "not greenfield."
Legacy projects are usually have shoestring budgets for any changes. After all, they are legacy for a reason: likely nobody has touched the system in a while. I can't think of a single legacy client I have that would be okay with a company wasting their hours on adding a new framework. It can be hard enough to convince them to let us add tests or upgrade from PHP 5.6
It's not limited to shiny new frameworks. Following naming conventions, design choices, etc, can take work, especially if you see something that isn't "best practice". Heck, sometimes you need to refactor something and the best thing you can do is reimplement any bugs you found (obviously with documentation and discussion to try to surface the why)
I used to work for a company where the American devs indented with 4 spaces, and the German devs indented with 2 spaces. When they finally decided to standardize the code style across the company, they couldn't agree on indentation. So, they compromised and everyone started indenting with 3 spaces. I swear I'm not making this up. I worked on one codebase that lived through that transition, and some source files would have all 3 indentation styles scattered throughout.
> breaking the standards of the project lead to higher mental load to maintain that code over the long term.
Nah. Over the long term the lava layer pattern is the only way to stop the mental load of the project from growing indefinitely. You will ultimately have to keep up with modern development practices, because keeping everything on standards from 2010 will ultimately be even worse, and the longer you leave your migrations the more painful they will be.
You don't and can't fully replace each layer, at least not in order - you should expect to have three or four ways of doing things in the codebase, that's just the reality of large old codebases. It still beats having your whole codebase be stuck on the oldest of those three or four ways.
Agreed, this is quite annoying, but there is a workaround. You can tell blame to ignore commits like this, through a config option blame.ignoreRevsFile or similar CLI option. Not the most convenient perhaps, but it’s something. I believe GitHub also supports this, though you may have to request it to be enabled in your repository.
Given this, I tend to prefer a single, formatting-only commit when introducing formatting standards to an existing codebase. Otherwise, it’s difficult to take advantage of QOL features like auto formatting in your editor, or other formatting tools which tend to operate on entire files. Then PRs end up being mixed with formatting changes, which adds friction to the review process.
FWIW, git now has options to ignore such commits. In a big codebase each big codemod sha should be added to the blame ignore file .git-blame-ignore-revs. It's supported by GitHub out of the box
Sure. Of course you can't commit to the main branch and your commit will never pass code review (which is a requirement for merging), let alone the automated tests in CI that apply formatting and fail if anything is not formatted as it should be.
These safeguards save so much time that there should be no excuse for not having them.
Not wanting pipelines to fail because someone put an extra space after the line, or because someone's text editor didn't put a new line at EOF, or put a unicode BOM or whatever is a pretty good reason not to have this "safeguard".
Just to be clear, the argument for this has nothing to do with how the code looks and has everything to do with how you can swallow a pill, once, and in exchange you get clean rebases forever. You can hide the commit from blame consideration.
I've seen changes like this drag out over a year because the product had so many permutations that thoroughly testing any change was an exercise in futility. I've also seen changes like this fail to happen just due to a lack of cultural desire to make things happen.
The first one is more forgivable - sometimes that's just the problem space. But neither of them are conducive to fulfilling working environments.
Might be true, but as someone who worked on a few legacy projects, the opposite, i.e. "You're not a senior engineer until you've worked on a greenfield project" is also true. There are skills that you won't pickup by working on old piles of code. Knowing what technology to pick, how to design your system for high-availability and how to avoid needless complexity are just as important to a well-rounded senior.
Perhaps in the end you are not a senior engineer until you've done a bit of everything.
Personally reading this thread I just realized that devs encompass so many areas. Requirement for senior engineers working on practical formal methods, real-time systems, Linux-kernel hacking, or those writing CUDA kernels all day are all gonna be different. System design is only for a broad kind, but not for all kinds, of senior engineers.
I agree with you. But, I have found the middle out effect in online discourse means we get articles (such as this one) polarising viewpoints for attention. Sadly, it works.
There's no hurt feelings like the hurt feelings of a junior engineer, who has spent the last year kvetching about how much they hate working on legacy junk, hearing someone else refer to one of THEIR projects as "legacy junk".
Any code that's old enough to have its first birthday party is "legacy", which means that "legacy" is a completely useless category. Anyone calling anything "legacy" is generally just showing their own lack of experience.
1 year old code is not legacy. Legacy is a useful category. COBOL is legacy. Maybe it is unclear exactly where to draw the line but if that were a valid reason to discard conceptual categories we wouldn't have any.
Heh, so true in many cases. I had rewritten a utility that scanned a directory and moved files to s3 from Perl to Go and eventually a different team took over the code. A bug popped up and they were not confident to update the "legacy code." I could do not but chuckle to myself: this was like a fee hundred lines of well organized and fully unit and integration testable code with a readme and runbook and grafana/prometheus and structured logs aggregated in Splunk. And that has been running for like 2 years. They just didn't want to even attempt the fix and in fact pushed it off indefinitely.
IMO legacy code is code that has lots of if statements and special cases sprinkled in over the years to enable new features quickly and get out immediate hotfixes for bugs without doing full refactors and/or data migrations to address the root causes. Even with 100% test coverage it’s a pain to build new features in because there are so many paths to think through, and instead of single sources of truth each part of the app assumes every other part is working in very specific ways.
An alternative definition, legacy code is any code where there’s no one left on the team who has been working on it for years and intuitively knows the pitfalls. Then everyone’s scared to touch it or make big refactors, which actually leads to those small special cases being added instead.
It's funny because 90% of my experience has been legacy projects, and the greenfield projects for me were the hardest, because it's really hard to organize as a team to build something from ground up, not kill each other with shovels in the process, and not introduce technical debt early on that will be impossible to pay off down the road.
When joining a legacy project, most of the hardest work has already been done for you.
Also, with legacy projects there is usually an array of helper scripts, classes and functions that do a lot of lifting. You have to rebuild all those as well.
Did those teams have lots of senior engineers? I’d assume when you have a single tech lead over each area the team won’t fight each other since junior engineers will just follow their lead.
There's also much more technical pressure on a greenfield project mainly because if it turns bad it's "your fault". There's no legacy to blame, no excusing bad choices (you made them!). It is part of the responsability but if and when it goes sour there's a bigger feeling of impostor syndrome than on a legacy project.
I started my career working on a large legacy project. It was the best learning experience, and super interesting. It certainly did not make me hate legacy projects. Work was like an investigation into who did this weird thing and when and for what reason - and there was usually a good reason. Then detangling if that reason is still valid today, or if we have time to refactor the thing appropriately to rewrite it in a different/better way, or how we can solve this obscure bug without breaking five other usages, etc. And doing all that as elegantly and non-disruptively as possible with hundreds of people on a tight timeline relying on the codebase. It was an extremely satisfying challenge.
I unironically like working on legacy projects. It's good to know it stood the test of time and my efforts spent on it won't be wasted, because it is unlikely to be dropped.
Of course this assumes there is a way to introduce improvements at all. Just keeping the lights on isn't as satisfying.
Legacy or not, I don't consider anyone really senior unless they have spent building and more importantly maintaining/fixing a system at least for 3-5 years at one single place and have done this at least 2-3 times across different companies. We all know that it is a lot easier to start a new/greenfield project with shiny tech etc. It is a lot harder to maintain especially once it is 2-3 years old and in production with tons of users.
The challenge there is that "legacy" codebase are just so much slower; you spend more time figuring out what needs to be done and how, than just writing code like in a greenfield project. And everyone knows the latter is just so much more fun.
I've run into a few engineers, very capable people and I'd be happy to work with them, but their experience was limited to very structured environments. It sounded like they were in well organized organizations.
One in particular worked for a sort of "side ways competitor". They were horrified to hear about my deploying a fix all by myself, no review, for a legacy product that really didn't have a good testing environment. It was inconceivable to them.
On the other hand, not long after, our organization of 4 coders put out a better product in 3 months than their massive organization did in 18 ... and using that some "inferior" legacy systems that just so happened to be better suited to the job than some fancy new framework.
It's not one or the other though - a bit of safeguards here and there would make you even more productive, imho. Code reviews are indispensable for sharing knowledge within a team, automated deployments help deliver the system reliably and without unplanned downtime, and tests... People start appreciating tests when the red lights warn them about breakages in some completely unrelated places that they didn't even think were affected by what they did. And I'm not talking about 100% coverage - even covering just some of the trickier, but important flows helps immensely.
I don't disagree generally, and in fact we're starting to build those safeguards where I am ;)
Having said that process and bureaucracy is still what they are and they tend to eat up more cycles as time goes on, even if only because they're administered by humans.
Legacy projects are great! If you don't have any current critical time pressures, you've now got yourself a beautiful puzzle to slowly unfold and understand. If you've got things like git blame annotations turned on you can read commit messages, see past pull requests and really start to get a feel for the people who worked on the project previously and understand how their minds work.
The biggest challenge is of course trying to prevent those feelings becoming contempt, rage and resentment as you see the same mistake repeated over and over again by one person who should know better while they make funny little comments in their commit messages while writing bugs that end up causing data corruption a year down the line.
The trick is mindfulness meditation and definitely not going to LinkedIn, printing out a copy of their face and then ripping it up while screaming.
I know a guy who always says you're not senior until you've had to maintain your own project that's become legacy and I agree. Too many developers serially job-hopping every 18-24 months. They all have experience with other people's legacy code but not their own. They're all happy to build something new but if they don't stick around long enough to see the consequences of their decisions, they're lacking critical experience.
I job hop a bunch and a common theme that I have had in my jobs is to go in and understand the legacy, and bring it up to a more reasonable state. It's been interesting having to be the one dealing with someone else's consequences.
I'll sometimes chat with the people who have had to maintain whatever ground work I had laid, and each time around, I get a little bit better at laying down a framework of infrastructure and operations that's better and better at staying sustainable
Refining/redesigning a process can be way more satisfying in the long run. Just have to remember to document those "avoided writing code" solutions for your annual review or to add to your resume.
Same here. My first job was “take this spaghetti mess and make it better”. It was an old crusty Perl CGI codebase without any framework and a ton of bad decisions.
This is usually doable until it's DEWS/SAGE radar code from the 50's. That's what NASA started with and the last guy died in the 90s. We got every other subsystem off the mainframe, but had to keep it around for the radar stuff until 2004 or so.
WEX/Display builder/etc:
Even when I was there my young peers weren't too keen on legacy/back-end understanding. GUIs were the cool thing. I worked on something that touched many legacy subsystems so got to work with a lot of "dinosaurs" (what the Pirates called them).
MCC testing was all AM shift, pirates/youts up front, retired controllers in the back. Test MOC took 30 minutes to recycle. I always sat in back if they'd let me.
In contrast to sibling comment: this is actually quite fun in small doses. Wouldn't like to have to do it for an entire product, but I've done some "competitive analysis" before.
I don't like gatekeeping, but I think a Sr should be a well - rounded developer. They need to have seen enough issues to be able to make solid engineering decisions.
I would have loved more detail in the "The limited role of developers" (or maybe merits a separate follow up). On the one hand, this section feels so obviously right. On the other, roles like ops and QA evolved for reasons so curious in wrangling with this legacy product how that could've been better / different beyond just consolidating work / roles.
"The only thing left was to deploy it so that we could test it. This was a problem, of course – in this case, we were not the ones deploying it or testing it. Deploying had to be done by the ops team, and testing by the test team. Why couldn’t we, as developers, manage the whole feature from start to production instead of opening tickets and waiting for other teams before we could close our task?
Firstly, we couldn’t avoid manual testing since much of the code wasn’t covered with tests. We also couldn’t deploy the application ourselves since the infrastructure didn’t allow us to.
This had us thinking about the reasons behind the separation of duties and how our current approach is better. Seeing that our lead and cycle time on this project was significantly higher than usual (it took us weeks to deliver something we usually deliver in several days), the evidence was strongly in favor of that."
One of my most educational personal project has been to update a ten y/o mupen64plus OpenGL plugin (Rice) to OpenGLES for RPi.
So much code that looks weird that I had dig in a lot and finally understood why this tiny piece of non-sense code is actually what make the whole system working, and think about how smart the previous dev was and how dumb I would have solve the problem myself.
I prefer a flat structure where you're a junior developer if you're still in training. I've held positions where my title was senior while I worked on a few projects, both legacy and otherwise. However, I think labels like these are harmful and create an atmosphere of authority and gatekeeping. I still learn new and exciting things from everyone, including junior associates. On the other hand, I know people who label themselves as senior who could be called junior.
After enough time, none of these HR provided labels matter and the work you do signals your level, not the other way around. Just a fair bit of warning though, regardless of your level, you should know how to implement the solutions from the ground up without corporate support and IT setting everything up for you.
For readers, programmers, and engineers stopping in here asking "ok what can I do to better myself here?"
I really recommend "Working Effectively with Legacy Code" By Michael C. Feathers. It is a top notch read, and helps newcomers (and vets alike) approach these legacy systems.
Marianne Bellotti’s “Burn It With Fire: Manage Aging Computer Systems (and Future Proof Modern Ones)” is another good book about working with legacy systems, though its focus is more about managing system evolution and modernization than specific code techniques.
A company I worked for had that classic problem of there being various older systems that didn't "belong" to any team and which nobody was confident in fixing or changing.
They did something which I thought was brilliant: they gathered a list of ALL of those projects and assigned every single one of them to an existing engineering team, whether or not that team had any relevant knowledge or experience (which was true for most of these older projects).
No team could complain about this because every other team was suffering through a similar challenge.
I think it worked really well! Bugs in older systems now had obvious owners, and teams were incentivized to dig in and figure out how the crusty old code they were now responsible for worked.
Projects are like people. Everyone shows great interest when they are being planned and brought to life. Some happen by accident and are more or less unplanned. Initially, everyone is enthusiastic about the new baby—or project. In the beginning, a lot happens, and things take unexpected turns. But after some time, things start to settle, and people's interest fades. Only a few loyal friends remain. Eventually, the project becomes a burden, and very few are willing to take care of it. In the end, many just want to let it rest in peace.
I find it incredibly satisfying to slowly figure out, document, tidy up, and very gradually morph a legacy project with the puzzle-like constraint of not actually breaking stuff or screwing over legacy consumers.
I think it’s less awful if your expectations are not that of a new project. Yes, there’s a ton of ugly bits at the interface level that you can’t fix. You make it as comfortable as possible while figuring out where to go next if anywhere.
We were lucky enough to experience explosive growth of tech industry in the past 20 years. A perk of this golden age is that one did not have to work on legacy code at all, if so we chose. Better, we got to build 0-to-1 systems[1] that could make a difference, like the first search engine, the first key-value store, the first machine learning pipeline, the first resource orchestrator, the first data pipeline, the first real-time OLAP system, the first distributed logging system, the first bespoke compiler, and the list went on.
Unfortunately, we had this level of luxury because we had growth, especially new business scenarios that involved thousands of companies and millions and millions of users, thanks to the confluence of internet, mobile, social network, cloud, and machine learning applications. I'm not sure we are going to have similar growth in the coming 10 years, but I hope I'm dead wrong.
[1] The downside, of course, is that engineers tended to abandon a project after version 0.1, like what Twitter engineers did to their Manhattan project.
Or a 6 paragraph Slack post in #general about what an amazing experience it was...to host a 30 minute Q and A on a certain topic. #community #growth #sharingiscaring
Well, if you are long enough in the industry, in retrospective, most of the code, frameworks and platform you have worked on are now considered "legacy", given the dynamic fast pace of our environments.
Data-centers -> Cloud
Monolith -> API and Services
Databases....
Memcache -> Redis
JS/JQuery -> MVP Framework
Git.
and not to forget K8S.
Honestly, it's a "know it when you see it" situation mostly, but there are some strong smells indicating there's a legacy project in the walls:
- none of the original authors are with the company anymore.
- it's written in a language/using a stack/using technology that is used absolutely nowhere else in the company anymore.
- it is big enough that migrating all of its functionality is a tremendously difficult undertaking, to the point where it's likely never going to get done, at least not in your lifetime (either in terms of your tenure at the company, or literally, before you die.)
Outdated architecture/code practices/patterns are a smaller smell, as is the lack of documentation/testing - but those can be present even in greenfield projects written by a big enough dipshit - and I say this having been that dipshit at several points in my career :)
IMHO 80 char lines are easier on my eyes, independent of monitor tech. I get antsy all the way over there on the right! Past 100 chars I am simply less confident in being able to pull off the ocular carriage return to the next line.
I find 80 characters to be too short, and the resulting code after passing through a linter is significantly harder to parse due to the indentation being all over the place.
Most engineers will scoff at the idea of patching up a legacy project if they see the slightest deviation from “best practices”. They will slam their fists on the table and claim that management keeps “piling on tech debt”. They will argue for a total rewrite and dismiss any concern of said rewrite taking years because “this is what it means to have high standards and best engineering practices”. They wear this as a badge of honor and frame the conversation as a question of morality and purity, in which they of course have the upper hand since they are not motivated by petty business concerns such as profit.
Engineers that refuse to acknowledge constraints, whatever the nature of those constraints may be, are not engineers. At best they are ideologues, at worst they are just incompetent. The most pathetic thing you can do is just continuously deny the laws of physics and reality, because it doesn’t suit you at some ideological level.
Truly elegant solutions are those that account for all constraints in the simplest, most concise way. It is those that do more with less.
I'd argue that even with automated tests it can be considered legacy. Easier to change things if the test coverage is comprehensive enough, but I've yet to find a project like that, even the ones I built myself.
I hope you find one one day. It makes such a difference to have a test suite which you trust, and which doesn't erupt in false failures at the drop of a hat. I completely agree, not all tests are created equal.
It is interesting, if there is a bug reported in the legacy system, the ticket can bounce between juniors and mids for months without fix in sight. It's like they don't have ability to read and debug code.
When it is something new, it is "easy" - check Stackoverflow or ask LLM and then modify code until it "works" - which doesn't even need any understanding of how it works. Just tweak until there are no errors and screen makes the manager happy.
Since corporations started cutting budgets - don't pay for training and get developers who know how to write CV and memorised interview questions and agree to work for substandard pay, it is all slowly going downhill. More experienced developers are tired of handholding new employees through basics or very much developing the tickets for them on top of their own workload and so they quit and problems are getting bigger and bigger.
It's a very interesting experience working in a large codebase that's a few years old while we're in the middle of trying to dramatically rearchitect everything because we realized the old way of doing things was creating tons of technical debt and confusing abstraction. Even worse, we're always trying to optimize things because we've got some soft real-time requirements and we're using about 50% of our entire time budget after we upgraded our bare metal servers to Zen4 (it was more like 70% before that). Every fix turns something else into the long pole that needs to be rebuilt with the lessons we've learned about writing clear, concise, high performance code.
As it turns out, if you're operating on large data in bulk you should probably just have a dense ID map and then do everything in arrays. Nothing beats arrays :)
I kind of love working on a legacy project, its like reading a love letter from past devs, their hopes and dreams, classes describing an abstraction that was never used. It's like reading hate mail from past devs, towards their bosses, towards ways of doing things they didn't like. It's like uncovering a stack of newspapers from a time before agile, when people had eldritch and archaic ways of naming tables and variables and methods, when they spoke of things we've forgotten.
Sometimes it is a pleasure and sometimes it feels distant and alien and frustrating.
Everybody hates working on legacy projects, myself included.
I am proud that I am a part of a team that uses most of the best practices
Speak for yourself... I'm the exact opposite. Then again, outside of work I fix and maintain other physical systems which are many decades old, some close to a century, so perhaps that changes the perspective. I see so many in the software industry cargo-culting "best practices" dogmatically, creating constant churn via useless trendchasing, that they keep reinventing - badly - new ways to do the exact same things as before.
> I see so many in the software industry cargo-culting "best practices" dogmatically, creating constant churn via useless trendchasing, that they keep reinventing - badly - new ways to do the exact same things as before.
That's why software engineering is still very immature as a science field. It's pretty much the result of so widespread software engineers' inability to even define the problem they're trying to solve. Along with the inability to demonstrate the impact of the change and provide at least some cost-benefit analysis.
And this is spreading like a cancer. Unfortunately, I see this almost everywhere and I am not sure when it all began because we were certainly not being taught such things at the University. Critical thinking seems to be lost in a signal-noise ratio.
Then I became a senior engineer pretty early - as my first job was working on nuclear power software. We had code in Fortran that NOBODY was allowed to edit. But lots of other legacy Fortran was up for grabs.
Oh, and my favorite software tale is when we turned on the new software in the control room and the first thing the plant manager said when he looked at the new digital instrumentation was "that can't be right".
I really like turning legacy code into non-legacy code. It feels like getting to solve a mystery.
For example, maybe there's really crazy hard to follow code and the person who wrote it left a decade ago and there's no tests so it's too risky to refactor it but you can write new tests, really delve into the use case of the code and then refactor the original code.
Or, maybe the code is using a version of a framework that's 9 years out of date but you can incrementally upgrade it after gaining confidence in any way you can.
i've worked on many legacy code bases and i think an underrated skill that you need is courage.
courage to be able to push changes you think could be breaking changes but always be optimistic that you to be able to fix as soon as possible. tests are ideal, but not everyone has 100% coverage. sometimes you just have to trust your gut and do diligence and push the change. don't be scared to push something because you don't know the behavior in production. the only way to know is to try.
Agree. But you need confidence on being able to debug / log / fix / revert, so having a strong knowledge of some of the parts is mandatory (i.e. server if it's web-dev, etc).
I'd also add "until you've spent some time testing a code base you didn't write" to the mix. That definitely tightens your senses and busts some assumptions.
The next time you hear the phrase "... well, it worked on my machine ..." it takes on a very different complexion.
My first "software engineering" scale project was working on a existing massive 10 y.o js pub/sub and java templated webapp. Taught me why typed languages are valuable, why I should never listen to PMs, why I should be humble, why estimates are impossible but sort of important, and why long feedback loops anywhere in the entire process (including compilation) are productivity killers, along with not having an isolated private work area.
I was being a bit facetious with that one. In retrospect, I would have pushed back quite a lot more, but for different reasons than I did when I was in my early twenties. PMs are there for a reason, but the realistic pace that a given change can be expected to be made on this type of software, especially as a new team member, is a lot slower than I should have tried to achieve and that they would have liked, and it's important to avoid burnout.
I thought I needed to earn my place and tried very hard, but slowing down and going home at the right time regardless of whether the thing I was working on was done or not, would have been smart. They placed a lot of pressure on me to produce perfect code at the same rate as all the java devs based on the assumption the assumption their job was harder because it was backend, but the tools and parameters of their work were wildly better at the time, and I had undiagnosed ADHD.
So, I'd be protecting my sanity more and telling them it'll get done next week instead of trying to fit one more thing in because "it's just a button" or whatever.
All projects I have done for the first ten or so years of my commercial work have been picking up an ancient code base long after the original team has left, and making it work. I reserved the nice and clean code for my personal hobby projects. At work, it was about getting the shit from the past working and maybe improve it a little along the way. You even adopt the coding style you hate, just to minimize diffs and making it easier to navigate the history.
> My feeling about legacy projects hasn’t changed – I still hate them (...) Instead of feeling resigned, we saw it as a place to ask questions and learn.
I see that you advocate embracing the opportunity to work on legacy projects, so I say even if you dislike legacy projects, try to avoid phrasing it so strongly. It might influence other devs in your team to develop a negative mindset toward them.
You are not a senior engineer until you work on a project from the beginning, leave no documentation, then leave the firm just as new assets are ramped up :)
This may be true, but taking on too much responsibility in a project or having to do those responsibilities will make you a senior developer. Otherwise, it is not possible to get ahead of the senior developer you are affiliated with.
A big problem I see in software is that developers don't stick around on projects long enough to see that their mistakes are mistakes. Instead, they move on to the next greenfield project or rewrite the project in the newest shiny tech without ever actually addressing the core complexity of the problem they're trying to solve.
Ditto Network Engineer who hasn't worked on an ancient ISP, ditto System Engineer who hasn't managed an environment with decades old hardware and software.
If you can open up vendor documentation and the default solution works in your environment you are either lucky, or have gone through hell to get it there.
All I have been doing for the past 10 years is deal with legacy. At least with the current one, the legacy is more recent and provides opportunities for myself and the other on my team to iterate on whatever we got, particularly in the realm of making operational improvements.
I still remember my first job porting a VB6 code to VB.net. The code was written before object-oriented code was a thing. Back then I thought OO would have solved it. Looking back all it was really missing was a strong type system.
I have a love / hate relationship with legacy projects. They're a fun puzzle, but often times the client has suddenly realized it's not possible to reach X and then I'm called in to make X possible. It can burn me out a bit.
Just because it's the latest kool aid tech, doesn't mean it's legacy; it often feels like it becomes legacy as the people pushing it leave.
But this is why technological choices need to be taken with care, and if you're in a managerial position, you should only agree on using new tech if there's dedication / guarantees for the ones pushing it, a healthy labour market, or budget to provide training for people that don't know it yet.
And that's why so much is still 'boring' languages like Java, .NET, PHP and Javascript instead of Rust or what have you.
I'm a Go fan though and will exclude that, as IMO Go is by design easy to learn by anyone.
A senior engineer is senior because of his ability to solve problems (that juniors created). Legacy projects are a lot like a suitcase of problems that you never anticipated.
I'd be happy to have a look at anyone's legacy code to at least write an analysis paper on it. (I'm currently seeking contract work, been doing code for a long time.)
and that's why I'm a senior engineer since my first internship!
I always considered legacy projects more interesting than zero to one. A lot less reinventing the wheel and bikeshedding, a lot of puzzles - painful ones, sure, but at least it's not a bunch of CRUD (or the usual startup experience, which is a constant flux of rewriting-crud-very-fast)
Does replacing a completely unsalvageable legacy project count? It's done and dusted at this point, but I replaced a C/PHP codebase written by an IP-trolling freelancer who kept project sources as his personal IP, won back after a legal settlement. It was all deliberately obfuscated single letter naming and misdirected logic paths, along with a heavy reliance on global state. Investigating any individual part was manageable, thankfully, but I still had to unwind a great many useless stack frames and reverse-engineer several intense procedural bit-logic algorithms.
Legacy is when you have a shell script call an ant script that calls shell scripts that call make scripts that fire c precompile requests to a remote database. (Of course it's now packaged in docker because it runs on a version of linux that makes the infosec guys come visit you at your desk)
Oh and every subsystem shares dependency references that are only used by one subsystem.
Oh, and the C includes are wrong and overwritten by the shell scripts and make files.
GOOD LUCK
OH, and your team is too scared to use the simplified build because there are too many things they don't understand, even though you did hash comparisons for every built artifact.
does everyone hate it? maybe I found my niche then because it’s actually pretty fun because you get to hack it up and come up with creative solutions, it’s like Indiana Jones exploring for lost artifacts, with all the perils
Could be fun with a decently made legacy system without artificial pressure from POs, managers, etc., but hell when you inherit a mess and nobody (who matters) believes you when you say you need to first fix 12 issues before you can do something simple.
Big code base. Big team. Big company. Different languages (and preferably in different paradigms; OO/FP and low/high-level). Different frameworks/stacks. And "in anger" also really helps IMHO (this means when shit went really bad and needs to be fixed fast).
Hard to disagree. I can usually tell what type of type of experience devs have by the snarky, dismissive responses I've gotten on various internet forums over the last two decades. e.g. "Oh but you would never have this problem if you performed proper code review" - Random_Rockstar_Dev_254
However not all legacy projects are bad to work on. If they're decently developed, then often you'll find that most of the pain is in setting up your local codebase. e.g. sorting out make files, or header file clashes etc. And if you're lucky, some poor bastard has already done the hard work for you.
As an aside, I know tons of experienced "Senior" developers who just suck at their jobs. The problem is they have tons of experience in delivering terrible products based on terrible code and architectural decisions. They just never had anyone to show them any better. And now that they're "Senior", noone can tell them anything. Many devs who work in corporates understand this pain. Shoutouts to my peers who have to "fix" 3000 line stored procs with a 100 line "change control comment saying stuff like "2009-04-03 JS Added TAX_REF_ID". I live your pain.
EDIT: Also, if you happen to work at company that thrives on terrible legacy products, try to drill it into their heads that BAU Support is not part of the solution. Every time I've raised the issue of the mounting tech debt I've gotten the response "Business does not have the apetite to solve these issues. And why would they? That's why we have support teams".
I see that going through these three kinds of projects let me grow as a developer:
1. green field project 2. other people's legacy project 3. your green field project growing into legacy project.
You can learn so much from each of these, but to me the most eye opening experience was our green field project growing into a project with more and more developers.
You could learn so much about others, some were very arrogant, went on constant refactoring mission only to mess up everything. If for some reason, I couldn't check what they did, usually, I had to come in and fix their stuff, but sometimes the only person knowing about the edge case was me, so I just left it "messed up".
Others tried to understand why the system ended up this way, some accepted it, while the best actually improved the system by looking back and recognizing the simplicity hiding in the mess.
> very arrogant
My experience, is that this is usually a defensive shell around personal insecurity.
On the outside, it looks the same, but internally, insecure people can be reached (not easy), whereas truly arrogant folks (a lot more rare than you might think) cannot.
My experience is that most difficult people are actually decent folks, that we can enjoy working with, but we need to adjust to them, and they need to adjust to us.
> > very arrogant
> a defensive shell around personal insecurity.
could just be despair and honesty.
I was reading about the kindle os here and wonder how many people are facing giant hairballs in their professional life.
> the Kindle’s OS is a mess of Java, Javascript, Lua, Native and React Native code. ...
> The Kindle runs stripped down Linux with a React Native frontend along with Java backend applications.
https://kindlemodding.org/kindle-os/
From your post:
> The Kindle runs stripped down Linux with a React Native frontend along with Java backend applications.
Honest question: what exactly is wrong with this setup?
There's the OS, there are services and user land apps written in java and misc native code, and from the link there's a single component where Lua was used as bridge code.
The GUI is React Native, which is just a framework to simplify the design and implementation of native GUIs.
So exactly what's wrong?
It isn't using their preferred stack.
Preferred stack: The one they know.
The meme image is very funny. Recommended reading: https://kindlemodding.org/kindle-os/kindle_os.png
I literally laughed out loud on the bus reading the linked file. The infographic is all too true.
> but internally, insecure people can be reached (not easy)
How? Any advice?
Well, I have found that finding things in common with people is usually a great segue into improving relationships. Learning more about them, also helps a lot. Find the things they care about. It can be surprising.
I have learned to make the first move, myself. It doesn’t work, if I insist on them doing it. Also, I need to accept their value system. If they truly have values that I can’t deal with, I usually “compartmentalize” things, so I don’t deal with them, in those areas.
I’ve spent my entire adult life, getting along with people that don’t play well with others. I haven’t always been able to get through, but I’ve had more success than failure.
> Also, I need to accept their value system. If they truly have values that I can’t deal with, I usually “compartmentalize” things, so I don’t deal with them, in those areas.
It feels like working with all sorts of people (and sometimes working around them) is an important skill that’s not usually focused on enough, which is how you get people that might be brilliant but also are really hard to work together with.
Not OP but: destigmatize admitting failure (by doing it yourself), normalize admitting ignorance and asking questions (by doing it yourself), find someone they look up to and demonstrate healthy collaboration dynamics with that person, explicitly labeling things which you value, as they happen (e.g. "thanks for saying you're not sure, or I would have thought that you were and it would have made me value the statement differently", "thank you for not taking the feedback on the code personally, the result is a better codebase for everyone"). And do the same with them whenever they imitate any of that, without dwelling on it any further or treating it any differently.
TL;DR: Normalize healthy dynamics by example.
Obviously just my 2c and every case is different etc.
To add (some overlap):
I would state that the first point can be a bit tricky. I've worked in some environments where one or two people follow the point and no one else does, so this ends up with those people being seen as "non-productive." This is more representative of poor management though and I'd push back against the toxicity to the extent possible.For managers: the last point is really important. Your coders shouldn't always be sitting in front of their IDE or terminal typing code. They should be spending a fair amount of time reading docs, writing docs, and googling. Writing docs is actually important as it helps communicate ideas, which also serve as reports to YOU. This communication also helps de-abstract the ideas and often leads to finding issues. It is an important step to debugging. (It is often a good idea to have juniors build or append docs when they are onboarding. Dual serves as the intro to the codebase)
One of the best ways for me to learn, is to explain.
That's a very old pattern.
That's one of the reasons I write stuff that no one reads[0]. Writing up a technique forces me to research it, and break it down into its simplest components.
[0] https://littlegreenviper.com/miscellany/
[0] https://fs.blog/feynman-technique/
[1] https://en.wikipedia.org/wiki/Rubber_duck_debugging
Be vulnerable yourself?
> Be vulnerable yourself?
Being vulnerable to an insecure, arrogant, asshole just means becoming a doormat. Ask anyone from an abusive relationship.
Being vulnerable is not the same thing as not having boundaries.
Exactly this. Strong back, soft front.
Ooh I like that one, I’m gonna remember that.
One of the places I feel like I grew the most, and I was lucky to have that early in my career, was freelancing. Every month I'd be ushered into a totally new project/codebase, with virtually no time/budget allocated to "understand" the codebase properly but still needing to deliver value (which was a stark contrast to my previous hobby dev time).
Wow this is a great formulation. I’ve started becoming senior more recently, for most code I’ve become pretty good now at making the tradeoff of how much refactoring to do versus a quick hack based on risk. Sometimes adding in an if statement makes more sense than a full upfront refactor.
The exception… is for any feature I added myself. If I spent a lot of energy releasing v1 with the “perfect” (to me) abstractions, etc. I get paranoid about more junior developers messing it up for new features. Re-reviewing code changes, insisting on all the upfront refactors and rare edge cases handled, etc. Something I need to overcome.
"3. your green field project growing into legacy project."
You do all this upfront design about how it's going to work and then "Oh god there's so many splinters and sharp corners, it's only getting worse and every new team onboarded to the framework needs to have their hands held, and management won't allocate time to address the tech debt."
[dead]
Yes! #3 is key to making one humble
Nothing quite like suffering through your own decisions to make the lesson stick!
Ugh, too right. Going through that made me appreciate working on a legacy project later on, because I could improve things rather than watching things get worse.
ive been having a blast looking into what parts of my greenfield turned legacy project are still around, vs the parts that evolved.
the new teams doing some cool stuff with my old baby
> 3. your green field project growing into legacy project.
This is always a fun one...
"Who wrote / designed this garbage... Oh wait, it was me"
> I can usually tell what type of type of experience devs have by the snarky, dismissive responses I've gotten on various internet forums over the last two decades.
I see this constantly on HN and have had to learn to not engage with those conversations. It is like some people have never had to run a project under any sort of external constraints like head count restrictions, layoffs, inheriting a system which you didn’t write, or awkward organizational structures.
Some will scream until their ears bleed that there is only one optimal way to do things and if you aren’t doing things that way you are a incompetent chump — totally disregarding the reality that people have to work with.
The real miracle is when you see a teammate who can still perform well under these insane constraints. I have seen it a couple of times in my career. They still find a way to write "good enough" code or write "good enough" docs. It was inspiring.
I would say the same about a sales person who can still manage to have a great year even when the (their) market is cratering. Again, I have seen it a few times, and it always impresses me... watching them pull a rabbit out of the hat!
Or the other typical HN solution to this: just leave that toxic company and go somewhere else! (Maybe it was valid during ZIRP, but not now)
It feels like you're just finding other ways to describe people who are senior in title only which is far too common. Any senior that uses their title to justify a 'you can't tell me way to do attitude, is a senior in title only.
I think the term is the issue. Senior development means something more intrinsic than it does in other title, like a senior manager. I think what we're attempting to define is something closer to seasoned developer.
> I think what we're attempting to define is something closer to seasoned developer.
I'm fully aware of that, which is why "Senior" is in double-quotes, but experienced (aka "seasoned") is not. My point is that you can be seasoned at delivering bad products. The point about seniority just speaks to tenure at a company. Sure, you can join a company as a "Senior dev", but that's not quite what I'm referring to here. One would think that they would be exposed during the interview process, but alas, we all know that's often not the case.
The title is more or less meaningless these days. That said the other problem is people can't always appreciate the perspective of a more seasoned developer. Some people who are very junior or intermediate think they know better since they don't have the experience. Lots of things in software come down to judgement/intuition and are not black and white. A lot of software development is about working with people and culture, not code.
The unseasoned developers are sometimes very keen on some idea, philosophy or method and are very intolerant of skepticism from someone who knows there is no such thing as perfection and no one way of solving all problems.
If I've learned anything in my history of coding it is that where the program ends up is never where I expect it to. My experience makes me better at predicting that differential, but to be honest, given a big enough project you are never really going to be able to predict the end state. Things change that are out of your control.
I'd also argue that this is why it is so important to write documentation and comments while coding. It'll help you later on and anyone that is onboarding (small cost to document, but many people reap the benefits). I don't want a junior coming in and completing a task fast, but I want them to come in and learn the codebase and how to adapt it for the task. Sure, it isn't as fast, but juniors are an investment anyways. I think we have to be honest that no matter what we do we are either investing or taking debt, both of which compound. But I think the problem is that interest is hard to observe and debt is less observable than the direct cost seen in an investment. Maybe a real Sr Dev is the one who is more aware of these trade-offs?
Always a favorite
https://news.ycombinator.com/item?id=11396045
[0] https://github.com/thehackerwithin/illinois/blob/5f91a29b1d4...
> As an aside, I know tons of experienced "Senior" developers who just suck at their jobs. The problem is they have tons of experience in delivering terrible products based on terrible code and architectural decisions.
It's a classic tragedy of engineering; you can actually ship with almost anything, and just successfully delivering a product reinforces a feeling that your choices were good, no matter how arbitrary they were in practice.
People can get really confident that what they're doing is tried and true, while it's simply the only approach they've had for all of their career.
> I can usually tell what type of type of experience devs have by the snarky, dismissive responses I've gotten on various internet forums over the last two decades. e.g. "Oh but you would never have this problem if you performed proper code review" - Random_Rockstar_Dev_254
Yes, and the flip side to this is the other kind of senior that you mention in your comment: the one that refuses to believe that things can be done better.
It's hard to find the right balance between wanting to write better code and pragmatically dealing with the realities of a huge, messy legacy codebase with tons of bad decision - you can't fix everything (and certainly not at once), so you have to choose your battles wisely.
The windows kernel is old. The Linux kernel is old. These are both old code bases, but they're not "legacy" at least in terms of how many would phrase the term. A codebase let to rot is legacy. A codebase that is constantly improving itself to be in the best state so that it can adjust to modern programming standards is just a good piece of software.
That all said, it's all subjective, blah blah the end.
Linux kernel is old rotting legacy which is improving itself.
Maturity and humility are key factors in being a senior developer. Those who lack humility will often cry about how its not their code that's messed up, delaying actual bugfixes and wasting developer hours, or worse producing a toxic work culture.
> 2009-04-03 JS Added TAX_REF_ID
Hey at least they’re using iso 8601 dates!
You can learn from anything. A "legacy" product is one that works, but is generally hard to maintain.
"Why is it hard to maintain?" is a question every architecture person should be able to answer...especially after digging into some large, old piece of software.
What if you work on legacy code projects but use AI for that ?
So, yeah, the minimum bar for "senior engineer" is very low. Can we all just accept that and move on?
Legacy code is amazing RAG material to throw at an AI ;)
[dead]
As an embedded engineer in a new position I always volunteer to take up an old crusty legacy product that nobody else wants to touch. It's a quick education into how the company approaches projects and all the ghosts they had to fight along the way. All the business logic is there, and sometimes it's even documented.
And once you have a semblance of knowledge about how the thing works and how to make fixes, you've built a small moat around your career there.
Additionally, reading through other people's code is educational and it also builds self-esteem.
Conversely, reading through my own code from ancient history (>3 months ago) is both humbling and terrifying :D
This happens to me all the time, but mostly with work code.
I'm under pressure, I have to deliver, things get rewritten many times, edge cases are found and fixed in production after they slipped through multiple qas but need to be fixed instantly or we lose money.
It happens.
But personal code? I am almost always proud of my work. It shows in general I was not under pressure and enjoying building it.
Even my code from the first year of programming, it shows it was made to make a point in writing a nice solution.
I may cringe at times, but still proud.
Context does matter, and code quality is more often than not a byproduct of our emotional state rather than skills.
I feel very similarly about the split between my professional and personal code.
Theres always something that needs to be rushed out the door at work and honestly nobody pays me for pretty code.
But my own projects? The code is a large part of why I do it .
Like a shiny coat of paint that blinds me, I scrape my code and write new in place, maybe this time with comments so I don't forget.
It's often "What was I thinking" or "What was I on" as Kernighan's Law bites you again.
I find this to be true in many aspects of life. Just watching how other people do things is often a huge self-esteem boost for myself.
A quip I'm often fond of that I think I stole from an old web comic eons ago
"I can't ever get even with past-me, and I'm already on thin ice with future-me"
I'm doing my best to prevent myself from being on thin ice with my future-me.
I posted recently about how I divorced not too long ago. I'm in an apartment now (for the first time in 16 years) while I stack up cash to build a new house.
I always think I'm not making any progress but then I see my next door neighbors ordering DoorDash 15+ times per week, and I realize that I'm doing pretty damn good for myself.
Or I walk by the mirror and think about how I've neglected my lifting for the past 6 months while I finished up this SWE degree. But I'll see those same neighbors struggle to walk up the stairs and I again realize that I'm making some awesome choices compared to those two folks.
ironically most people can say the same about coming back and reading their own old code
I forget who said it “The worst programmer I’ve ever seen is me a year ago.”
I found it true for the first five years into my career, now it’s still true but with a longer time scale, which I suppose is progress
In important regards, you sometimes become a different person :-)
"No wonder that didn't work."
> it also builds self-esteem.
Well crafted dig.
Imposter syndrome is fueled by colleague deification.
It's easy to view smart coworkers as superpeople... until you're reminded that they, on occasion, make dumb human mistakes too.
I’ve found one of the best things I can do to help my jr devs feel more confident (and be willing to ask for help) is highlight my screwups and “basic” things I’ve just learned. It make me seem more approachable and less like everything I say is a pronouncement from on high, it makes them more willing to admit to their own mistakes, it clarifies that we’re all learning stuff all the time and it helps spread knowledge about things that folks might otherwise be too afraid to ask about for fear of looking silly.
A benefit of being a sr dev is getting the benefit of the doubt on mistakes, so I might as well use that to my advantage and let everyone learn from them, not just me.
I 100% agree with you, and have used the same technique. Showing juniors that you are also human and make mistakes allows them to be less anxious about learning and trying new things. It also helps to keep them from trying to hide their mistakes which can quickly lead to larger system issues.
Of course, you have to have a culture where making a mistake doesn't immediately lead to being let go or publicly reprimanded. I have definitely learned quite a few things about people and code by helping juniors work through bugs that may not have been so obvious (including occasionally finding bugs in 3rd party libraries).
Pitching in to your and parent comment but from the junior side to say that yes, this definitely helps :)
This is a fantastic insight.
I used to suffer from imposter syndrome significantly. It helped to see the mistakes others made which anyone could have made. It also helped to have my own good ideas and talent explicitly acknowledged. Learning to not take criticism personally and own your own mistakes also helps to rob imposter syndrome of its power because it demonstrates to others that you can be forthright and self aware, and that helps build trust.
I find that I have to argue a lot for making systems where humans are allowed to be human and not burn the house down.
Operationally, people gravitate toward trying to treat the humans as machines.
We have machines for that.
Sure, but if there's anything I've learned from working on legacy software, it's that I don't know why the code was written the way it was. I'm not the only smart person to work on a codebase, and it's more common that another smart person did what they did for a reason, than that I'm a genius who knows better than they did.
Sometimes there were reasons, but they were not necessarily good ones. Like these:
- time pressure, leading to just making it work and not caring about edge cases, which were bolted on later. Or maybe never at all. And naturally, there is no useful testsuite.
- the prototype ended up in production
- a refactoring happened, but there was no follow-up into other areas of the code base
- features changed, but the code is still influenced by the necessities of the old way it worked. Similarly, for changing technical circumstances
- code was copy-pasted or was worked on by too many people
- code was herded too long by the same person and now, to put it nicely, bears traces of their bespoke coding style
Chesterton's Fence is still very relevant, therefore I'd still err on the side of writing a lot of high-level tests before touching anything.
And not to forget: business realities change over time, but it is not always the code follow.
All of these, I have been guilty of. Fairly recently, too.
Kind of light fighting against the human tendency [0] to blame the failings of others on their core character rather than the situations they are in, and vice-versa for oneself.
[0] https://en.wikipedia.org/wiki/Fundamental_attribution_error
Such a rare opinion on HN unfortunately. But you're correct.
I don’t think there was any dig implied. And I will second that it will add to self esteem. Specially during discussions if you have more understanding about the code you will have both better contribution and take away
Sure, when you succeed in understanding someone else's legacy codebase, it builds self esteem. When you fail attempting to do that, it does the opposite.
I have inherited projects where I just could never understand what they were thinking well enough to make the service good (not running out of memory and crashing). It was very humbling; I still sometimes wake up in the middle of the night with things I wish I'd tried, but the assumptions of how it all worked was just scattered throughout a bunch of files and the assumptions of everything was just not how I'd do it.
Also why I used to like interviewing.
Self-esteem that easily turns into hubris though. I think the real seniority shows when you are able to work on a legacy codebase full of the shittiest code and not have the slightest desire to rewrite it all.
I think that value comes from code that solves tough or complex problems, which can be humbling while improving your understanding. Feels good when you get a grasp of what’s going on and can understand that there may not be a simpler approach, just a different one.
There’s also legacy code that is not great and is not solving a complex problem but is just a product of the conditions at the time. It’s easy to look at that and assume you can do better given the same constraints, and this is neither educational nor esteem-building. Just an ego massage.
Unless the "other peoples code" actually turns out to be yours that you wrote years ago, and forgot about :D
git blame is the best teacher of humility.
Self-esteem and empathy.
Haven’t you also often made a moat between yourself and doing literally anything else more interesting at the company?
Engineering isn't about working on the most interesting problems. It's about getting stuff done and management happy.
Here, parent explained in detail how to get stuff done, management very happy and secure their position for years to come.
> Engineering isn't about working on the most interesting problems. It's about getting stuff done and management happy
Truth is harsh, however this seems to be 100% accurate for nearly all cases of employment. Rarely do you get to focus on simply interesting problems and good engineering as a primary concern
Boredom is in the mind, not the task. Things aren't boring, people are. An important type of intelligence is the capacity to find what's interesting about a task that others lack the imagination to see. One needs to be able to create their own interesting solutions rather than expecting them to be handed down on a plate.
> find what's interesting about a task that others lack the imagination to see.
word of warning from an old guy, don't create problems for yourself when you don't have to. Turning something boring into something interesting can have painful consequences down the road.
How?
Boredom is lacking stimulation. Even the most cutting edge task can grow boring if you need to plumb with it dozens of times over a year. Just because you can find what's unique doesn't mean it stimulates you. That's the exact issue with why neuro divergent individuals are demonize: they don't take as much interest with people as "normal people" would.
>One needs to be able to create their own interesting solutions rather than expecting them to be handed down on a plate.
You should tell that to every old manager I had.
> You should tell that to every old manager I had.
They sound like boreful people. If you find you have become boreful, there's a good chance you may be experiencing burnout or depression, which are nasty diseases, but still ones that afflict the subject, not the object. Nothing is boring but for the person who perceives it that way.
>They sound like boreful people.
They have a job and family at the end of the day too. Not everyone has the power nor passion to try and and move a billion dollar corporation to care for people over profit.
>If you find you have become boreful, there's a good chance you may be experiencing burnout or depression,
I haven't has a full time job in over 18 months, so I hope I'm not burnt out.
>Nothing is boring but for the person who perceives it that way.
That's like saying nothing is ugly except for what people think is ugly. Boredom is personal and shaped by each person's experience, so I don't think framing it as a person for not trying hard enough is going to go too far in practice (nor is it productive to judge people based on how they prioritize their lives). Of course some things will be boring to one person and a life passion for another.We don't have enough time nor energy to try and find appreciation for every object and concept on Earth.
you should be a production line manager.
Is this why I love Factorio so much?
I’ve got some mulch in my yard that needs counting.
Everything in this comment is gold.
> Things aren't boring, people are.
What a strange thing to say! Aren't people also kind of... things? Shouldn't they be at least as interesting as things?
You gotta go into R&D if you want to focus on the fun stuff without the annoying plumbing. But such positions require an entirely different pipeline from getting a SWE position out of college.
A good way to secure your position is to be the go-to expert for a product with many years of life ahead of it.
Fixing stuff on a legacy product may make management happy but if that product is discontinued next year then you haven't accrued technical expertise valuable to the company (but you may have built a reputation as a fixer and quick learner).
So, as usual, it is a balancing act.
Edit: this is my perspective from the embedded world. It probably applies generally, though.
I'm referring to embedded, not web dev.
When times get tight the new projects get shitcanned and the 10 year-old cash cow design gets the promised new features.
One crusty project I worked on was a legacy control board for a piece of restaurant equipment. The customer, the company that built the actual machine, had been building this product for 40 years. It had been through two PCB redesigns and two different microcontrollers, but the logic was tried and true and had to survive. A port of the project from 6800 assembly to C had completely gone off the rails and the contractor was dumped. All it took was a 20-opcode fix to a routine that the contractor just couldn't grok.
It is indeed, but at the same time, things that have stuck around tend to stick around, and hot new things have a way more variable longevity score.
I wouldn't say that's the conclusion. If there's only one true thing about work is: management doesn't care about you. They can fire you for any reason, and thinking that by working on stuff nobody else wants to work on you are "safe", it's an illusion.
If any the conclusion is: work on what you want, life is short.
Engineers have a special place in society like doctors and lawyers. Working with management is part of the job, but engineers have a professional ethical obligation to say no if they are asked to something against the public good.
The split there isn’t in favor of doing stuff that’s fun and novel though; actually, the engineer should usually pick a boring proven solution if the public has a high stake in the outcome.
Only valid when engineering something which can kill/hurt/pollute. These areas are regulated and can carry personal liabilities when it goes wrong.
But the vast majority of HN topics are not concerned. Additionaly, it does not pay well
> Engineering isn't about working on the most interesting problems. It's about getting stuff done and management happy.
That's a perfectly reasonable thing to want out of engineering for yourself. I wouldn't state it as an absolute truth for all people though.
Personally, I'd like to be working on something that extends the state-of-the-art a little, even if only by a tiny fraction. It can be one for the other disciplines involved - it doesn't have to be the software I'm writing that is responsible for that (and it usually isn't), but that's what I derive satisfaction from.
It depends on your goals, though. Depending on the company, that can be a very good way to be treated as a cost centre to be minimised.
Yes, let's minimize the only dev working on legacy software.
Do you think that software becomes poorly understood and maintained because the company treats it as a prestigious job and rewards people for working on it?
This is how I got laid off. Working on legacy software, sole person on the team, eventually management decided that it could be replaced by AI or some such pixie dust.
Legacy software with a single dev can be on the fast track to getting shut down. If it was still a business priority, they'd be throwing more resources at it.
Has said many a manager who thinks developers are fungible.
I don’t think the parent comment is saying that should be the case. But the way that corporate culture works, it is definitely a career killer.
That's a very narrow definition of engineering. And while it's not wrong, it's absolutely more of a "management" POV. Like sure, for management, engineering is mostly about what you said, but that's it.
Well, that's just, like, your opinion, man.
Depends on your definition of interesting?
I find improving established code of far more interest and reward than greenfield. Coming to understand what a person was thinking while building something is fascinating. My updates are tangible life improvements to real human beings. That interests me, at least.
Greenfield, I have the incredibly stressful job of making all the major architectural mistakes people will complain about for years to come. I'm not interested in that. There's also a much higher chance with greenfield that the product will just fail and no one will enjoy your work.
I’m interested in finding the patterns and architecture that prevents those mistakes. I think I’ve found it already.
You've found a way to correctly anticipate new user needs and new technologies years, even decades, in advance?
And you've found a way to completely forestall the non-technical demands that force non-ideal design choices?
You're sitting on a gold mine friend.
I should say not prevent it. But minimize it in the maximum possible way.
I am sitting on a gold mine. But I can’t prove it. That’s the problem with this stuff. The industry moves horizontally with each design such that nobody knows if say the current design trend was better than the last. We go better or we go worse and nobody knows if it actually was.
Maybe I am sitting on a gold mine. The problem is you’d never know about it. You’d never believe it. Because a proof isn’t possible.
So because it’s not provable you have no clue about any design whether one is better or one is worse and thus you disbelieve everything.
The overall question is how do you create a design or pattern that has maximum adaptability to anticipate any possible requirement change with minimal rewrites? Where you only swap in and swap out modules without changing structure?
I think I found the best possible way. But you won’t believe me.
I'm not the OP, but I'll say that I don't believe you, because you're telling me you have The Perfect Solution, without providing even a hint of what that solution is. If I was skeptical at first, I'm now certain you're selling smoke & mirrors until you provide any evidence at all you actually have an idea here.
I literally said I don’t have a perfect solution and that you wouldn’t believe me. What I have is the best possible solution based on available solutions that we know of. I found the best way.
I understand what you are saying. Obviously the type of development you do is a factor, as well as the way you and/or your team does development. I think others are looking for the one true solution, and you are saying that you have found the solution that works for you, given the way you develop and the constraints you deal with. At least that was my understanding of it. If you are doing embedded development, being thrown into a large front-end development project isn't going to use the same solution.
I have a solution for custom content management systems that usually makes it fairly easy for me to incorporate customer requests without needing to toss out my existing solution and start over. Almost a type of framework built over an existing open source CMS product. I maintain about 90 websites as the only backend developer, and have spent about 12 years now working through various versions of the software to custom tailor a solution that works for nearly all situations I have come across. I don't do front-end development any longer, but I was involved heavily at the beginning stages to make sure both the front-end and backend were flexible and broken apart into modules.
No im saying i have a general solution for everyone. The general part only applies to the part where are software projects are similar. The part where they are dissimilar it can’t apply. It’s impossible to generalize over distinctly different things but I found a solution that generalizes over what’s similar in the overwhelming majority of projects.
If you find a templates solution works for you then that is not a general solution. It’s better for you but you are hardening and becoming opinionated on certain details and that works for what you’re working on. My solution is a lower level pattern that is more general and less opinionated. For what you do it likely is only effective if you are required to drastically change an existing project while maintaining as much existing code as possible. The solution prevents a complete rewrite and allows you to modularly pull out what you don’t need and replace with the things you do need.
If you don’t need 90 percent of your project this pattern won’t prevent that. What this pattern allows is for you to actually pull out that 90 percent. Many projects that only need you to pull out say 30 to 40 percent of it actually end up with a full rewrite because the modules aren’t decoupled enough.
Apologies for the misunderstanding. Sounds interesting, but saying you can't prove it, and nobody will believe it is a tough sell. I guess I find it hard to believe like others, but you are talking in very abstract terms, and not being able to (or refusing to) give any kind of details sounds like you are trolling or don't realize that your solution does only work for what you work on. I'd say I could be proven wrong, but it seems you don't wish to do anything other than generalize. Good luck to you if you've actually found what you describe.
No need to apologize. I assure you I'm not trolling. Yeah I am talking in abstract terms because I don't really want to get into a debate on the topic. The reason is because any design style in this area is basically unprovable and leads to endless debates. There's no point.
I do have something that explains my pattern and serves as support for it. But it's no proof because such a thing isn't really provable. Anyway it's easier to let AI explain it. In the following conversation I basically ask chatGPT about this topic with very general questions while manipulating it as little as possible.
https://chatgpt.com/share/67b3ca4b-fc5c-8001-a072-342357b8a6...
chatGPT basically arrives at the same pattern I discovered. Where I cheated is when I started giving my own opinions on the several patterns chatGPT suggests and I hinted the LLM at the direction I wanted to go. Everything else is a very general question.
That being said, there are clearly other sets of general questions that can very well lead chatGPT to form alternate conclusions, but this one aligns with what I and many others have discovered.
Those are pretty well known FP oriented guidelines that many developers here on HN are familiar with. I don't think that your fear of not being believed is warranted.
HN is mostly not hardcore fp programmers. There’s a good amount of fp programmers but they are not the majority. Plenty of go and c++ lovers here and people who don’t understand the guidelines above.
The point is I don’t want to argue about it with someone who doesn’t get it.
Check back in a couple years ;)
I did it. There’s always things that went wrong. But I found the way to minimize it.
I pulled this strategy at a new company and found that it opened up a million opportunities I could take my pick of. I basically got to shape my own role going forward.
Why? Because I very quickly made a name for myself as someone who learns quickly and then gets stuff done even in contexts where everyone else struggles. Solving problems that have long been considered intractable is a good way to show your value.
This is 100% true. Those problems you are solving probably don’t look all that important, but, in reality, they have been blocking sales, renewals, account expansions, or even access to markets for a long time. The business side of the house has probably given up or occasionally tries to wag the dog to get it fixed.
> Haven’t you also often made a moat between yourself and doing literally anything else more interesting at the company
Nope, assuming that code works and handles UTF8 (sorry, but there is a point where it is too far gone to save). You’ve made yourself essential. You know the business better. You are strategic. And often, you become the one person who knows the legacy code base. And you will quickly find yourself a part of super important decisions. You’ll be called out regularly for fixing things that have gone unfixed for years. You’ll also drive value by helping the company avoid spending vast amounts of money re-implementing functioning code. On top of that, new product is often a mix of new and old, so you’ll quickly find yourself in the middle of new product teams.
These insights are essential for playing the long game; it may sound distasteful for people early in their careers. That’s fine, but when you’re running out the clock to retirement, can’t go wrong.
It's not just about job security.
Even at a young age, some people find it super fascinating to work with old stuff and study the traces of people long gone.
If you're that kind of person, digging into an old assembly codebase is going to be so much more fun than learning some newfangled web framework.
I have hobbies.
The most interesting work is going to be 10% interesting and 90% not interesting. In most cases it will be less interesting than that.
Prioritizing "interesting" work means you'll always be exploited by the companies that can sell you on less pay, less benefits, less time off, less respect in exchange for some pittance of perceived novelty. Don't fall for this trap.
Ex: Game development.
tell me you don't have ADHD without telling me you don't have ADHD
For what it's worth, I find it much easier for my ADHD to debug and improve existing code than to greenfield - it's actually interesting. Debugging is like a detective's work with small dopamine hit at each step of unraveling the situation and a huge reward once you finally grasp it. It's best to do on someone else's code, so you don't get too frustrated with your past self ;) Greenfielding provides some rewards too, but once I'd greenfielded anything remotely similar already there's a lot of forcing myself to do it around the beginning and end, where there's plenty of unrewarding work to do. For greenfielding to be fun and easy, it needs to be a completely new area that I had no previous experience in, but that doesn't tend to end up with excellent results.
On the other hand, the infrastructure variant of this is the exact opposite. Trying to figure out if something specific is in use and who uses it or if someone just clicked around in your cloud console and forgot about it is the worst. There's only so many ways I can be inventive in building proxies to measure usage.
We sometimes get incidents that things someone remembers should exist do not work, only to find out that that product was retired several years ago (but someone forgot to delete some DNS records, so there's an error message from... someone's server!). But only because we asked half the company on Teams.
Tell me you use AWS without telling me...
... I literally had a multi month ticket open to track down an expense for something I either never used or only used once which ended up being a load balancer or some crap
That is a good point and I'd likely be good at that.
In a world where everyone has ADHD, I welcome these few non-ADHD aliens.
Never said I didn't!
Actually you're completely wrong. I do have ADHD.
As a senior engineer your job is to take on the unglamorous hard problems, both technical and organisational, that are holding the team back. Doing this, and being seen to do it, early-on in your time with an employer will be somewhere between appreciated and expected.
If you do this you should also be good at story telling. Not working on the new things, but making sure the core business running, is often times looked down upon in my experience. Most leaders will see the greenfield project move fast and wonder why the established and "complete" software moves slower.
Yes, thats a very good point. And one that I learned to my cost in a previous job.
What could be more interesting than code that has managed to survive in the trenches of production?
> doing literally anything else more interesting
I found that working on legacy projects is often the most interesting.
There is the challenge of understating the original code base, which may look messy, but is full of history, you see the architectural mistakes and their consequences, how they managed to fix obscure bugs, tackle unexpected use cases, different code styles,... Then they ask you to implement a feature or fix a bug, you have to find what can be touched and what can't, what is a bug and what is a feature, see how much time you have, and decide on a strategy. This is production code and people are using it right now, and they are efficient with it, you have to help them and not force them to relearn everything for no good reason.
And there is technical debt management, your goal is to make the code better or at least not worse, but without introducing breaking changes and under time constraints. You have to manage your off time to make improvements that will help you in the future (chose wisely) as the next truckload of tickets arrive. It includes having a good relationship with your managers as no one wants you overworked, as it would just cause technical debt to accumulate with no time to repay it.
And good luck getting an AI to do that, if you want job security in the age of LLMs, that's your change. It doesn't mean you can't use AI, but you certainly won't be replaced by it.
More often than not it’s not enough work to just work on the legacy system that keeps chugging.
Nope. If you're willing to take on the shitwork without a lot of handholding and show real progress (which actually impact earnings) you'll have a really good stab at new opportunities.
Small, but not insurmountable. The time always comes when a new leader and their lieutenants come into the org and prioritize new features over maintenance, at which point your moat is null and void and you’re on borrowed time. I’ve yet to find a way to build a moat that withstands that level of hostility without actively harming the organization in the process.
I’d love to hear how others deal with this inevitability.
When you're on a reasonably large legacy project that's a pipe dream.
One of the old specialty C ERPs I've worked on has literally buried more than 3 (the ones I know of) "great Ruby rewrites". There are just _so many_ features, that the newcomers always ended up on the chopping block after 5-10 years and before they could achieve real customer traction. The old project just ended up getting some more interface fixes instead (aka even more features to catch up to) and to this day the compiler for the core hasn't even been upgraded to a version that can run on anything newer than XP SP1 (the improvements in SP2 break it).
Now they're doing another modernization project with some web services and stuff. We're happy to take their money for those as well. I'd be quite shocked if those do anything other than enlarge the moat of the core.
Yep. The only possible way to rewrite such projects is to do it gradually, piece by piece. It is also important that the refresh happens from within and is led by those who understand the legacy codebase and, especially, the business requirements it fulfills. No way can an external team do something like that. Being given project description "it must do everything the legacy project does, plus these new features" doesn't help either. If you get a ticket like that - run.
I just spent 5 years of my life building such a system. Then we were asked to do it again (a refresh of the refresh, if you will) and everyone marvels that suddenly we can do everything both fast and correct.
Well, we just spend 5 years training the team on how to build the old refresh, and now the new refresh can be done properly.
> The time always comes when a new leader and their lieutenants come into the org and prioritize new features over maintenance, at which point your moat is null and void and you’re on borrowed time.
The "on borrowed time" is an odd conclusion.
When someone takes over, it's typically a priority change ("We want everyone working on new projects, just freeze this old stuff") rather than some kind revenge path against anyone who worked successfully on the old stuff when it was a priority.
And a good engineer who is able to handle legacy code will typically be even better when working with greenfield code, because the latter is easier, all else equal. So unless you are refusing the priority change, why would you be on borrowed time?
It does happen that the new leader ends up saying out loud "everything we have is terrible," which is easily taken as "everyone who worked on it is terrible." And to be fair, sometimes that's true. And sometimes it's not true but the leader wants to bring over their crew of loyalists and needs to make room somehow.
Long-term ownership and careerist jockeying don't go well together.
My recent experience is that a lot of the system was not as good as it could have been because it's a hard problem. The company got bought out and all the new bosses come in and say all the existing code and way things are done is crap. Then, a year later, with everyone sacked and new people brought in, once again, these guys say the same thing and promise they are the top guns to fix it all up. I just checked out on the third round.
Ah, that's definitely true. But that feels like an orthogonal concern to you being an existing dev working on legacy stuff vs greenfield stuff... unless the new boss likes the greenfield stuff for some reason. What you are talking about is more of an ever-present possibility of "new management fallout" in almost any field.
if you've built up respect at the company and speak your mind regarding the maintenance vulnerability, the new crew should pay attention. if they don't, then you're screwed anyway, but so is the company.
source: endured this while working at a startup that was bought out by SFDC. Kept my principles, but lost the job. /shrug
Yup, you and Shermantanktop get what I was asking perfectly. Seeking confirmation I hadn’t overlooked anything that I could’ve done better, but you’re both dead-on.
The irony is that for those of us who enjoy maintaining, improving, and eventually replacing legacy stuff (code, systems, infrastructure, etc), what we really want is often just stability in career - hence the moat. It means it’s cheaper to keep us around for the long haul than rehiring us again at current market rates every few years, especially if we’re making ends meet with a high quality of life. But that’d mean focusing on the long term, which I don’t see many companies doing in general.
This can go wrong, as I found. Management and Tech leadership are focussed on the new projects with latest technology - they pay the biggest bucks for these and no one really cares about the devs maintaining the old stuff - even if important. The second problem is looking for new roles you resume is filled with old tech rather than new stuff that is in demand.
I’ve found it to be the opposite but it’s not really either/or.
Over here in the UK, if I constantly chase the new hotness I will have plenty of new startups to choose from but I’ll almost always be low-balled on pay with some practically meaningless equity package.
Gaining more in-depth experience opens up the better paying positions further up the ladder in more established companies.
Don’t want to keep your head in the sand for years but at the same time, there are a lot of skills that can’t be learned if you stay exclusively at the bleeding edge.
The moat is real. A shiny new project can always be cancelled, but withdrawing support for a suitably large legacy application that doesn't have a replacement is harder.
And if you want a boat for that moat recommend a modernisation project for the existing philosophy.
Also, some companies and teams do versions of this, officially or unofficially.
One pre-FAANG tech company I worked had a policy (known among the local schools) that each new hire has to "pay your dues" on grunt work, before you get to do fun/glamorous work. That kinda worked for them.
A problem they had was that they were doing desktop development in C and C++, which, among other purposes, had to ship software bundled with new brand-name PCs (so not necessarily able to bugfix over Web later). The real problem was that very few new grads are minimally competent at production-grade C or C++. But, in early Web years, the company couldn't hire enough experienced software engineers to relocate to a college town. So they had to try to train and retain local grads of a top CS program. That might've been part of the thinking behind "pay your dues"... before you get to make rookie mistakes with greenfield.
Similar here. First thing I do when joining a new project is to fix bugs in a legacy product. Forces me to read through the code, follow the data and execution. Nice thing about bugs is also they (mostly) have defined behavior.
>embedded engineer
Important note: building the moat means you can one day be promoted to principal and make junior webdev money
That's insightful! Especially the part about "all the ghosts they had to fight along the way."
It's all fun and games until eye̶s̶o̶m̶e̶o̶n̶e̶ ̶l̶o̶s̶e̶s̶ ̶a̶n̶ ̶e̶y̶e̶ management demands some new feature and refuses to understand why it would take so long to implement it on such a stable and well-established p̶i̶e̶c̶e̶ ̶o̶f̶ ̶o̶l̶d̶ ̶c̶r̶a̶p̶ project.
Good advice, now I’m going to have to figure out the office printer, and find my linkedin account…
Smart move. The new guy always gets what no one else wants so you're going to get it anyway.
Cannot recommend this enough, this mindset will set you up for success.
and forgo climbing more in the org because you become indispensable in that role
Agree. Also, not be judgemental about the state of things as in "which moron did this". Approach it from a point of understanding and respect rather than criticism. There are multiple reasons why the code base and tooling around it are in the crusty old state. They are usually decisions that engineers or middle management had no control over.
I've been with my company for a while now. Too often I find someone come in with the "this is trash and all of you are idiots" mentality. I've typically not been keen to help such folks. They usually soon ask for rescue from the very same dumpster fire they created.
... just don't make the mistake of documenting too well :-)
Developers need to realize that AI means that a lot of coddling our profession used to get is going away. Employment has always been a tenuous relationship with a fundamentally sociopathic entity, but things will be getting a lot worse in the future.
[dead]
I once had a manager that was fond of saying:
> Experience is what you get when you don’t have any.
The only better experience than working on a legacy codebase is working on a greenfield project long enough to watch it become legacy and see the good and bad consequences of past decisions.
And the only better experience than "working on a greenfield project long enough to watch it become legacy and see the good and bad consequences of past decisions" is working on a second greenfield project long enough to see that drastically overcompensating for all the bad things from the first one is not the right solution either :)
Fred Brooks has a whole chapter, 'The Second System Effect', on this in 'The Mythical Man Month.'
"The general tendency is to over-design the second system, using all the ideas and frills that were cautiously sidetracked on the first one."
Brooks reasons that the combined experience of doing the first project well and the second project badly leads to better designs from then on.
And the only better experience than that is killing a new greenfield project before it gets to production because the old software was good enough and the problems were organizational in how the software was being used.
The best code is the code that doesn't exist.
Certainly the most secure code. Not sure about best.
Better on every axis: security, performance, resource consumption, reliability, verification, documentation...
Code is a pure liability that you accept to get a useful service.
exactly..which is also why lines of code written is one of if not the worst possible metric of productivity. easy to game too.
I marked my transition to senior engineer when my net lines of code flipped to negative.
Not that I game that obviously, it just occurred naturally for a ~4 month period
> Better on every axis...
How 'bout job security?
Anyone can write no code.
But it takes a master to not write the write piece.
I wish this happened tbh. I've seen one where the greenfield (Scala, AWS, etc) is still living alongside the old good enough software they went back to (C# / .NET) ten years on.
I thought it was pretty funny how we had this large project that was supposed to replace a legacy system (that was mostly a bad hack that got pushed to production).
But when it was finished it failed to meet the basic requirements for the only customer that used the system.
Once you're senior enough those cases won't be funny any more, just sad.
I have a very dark sense of humor sometimes.
But yes, it's sad.
I keep telling people v1 of anything always sucks.
One of the tells of an inexperienced engineer I use is how much they disparage the previous team's work.
> how much they disparage the previous team's work.
And many fail to discern between "disparage" and "critique" or even "Question in order to learn"
One of the greatest failings I've seen in leadership in our time is the idea that in order to make a critique one must come with a solution in hand. As a leader I want to know the things that are going wrong as soon as they're seen, not to require someone to go through the heavy lifting of a solution before they say a word. Now, of course, there's a difference between bitter unhelpful cynicism, and simply identifying a gap between the current state and optimality.
> And many fail to discern between "disparage" and "critique" or even "Question in order to learn"
I think I’ve had the conversation with new to my organization devs a few hundred times: “Look... saying code is crap or stupid is telling others you’ve given up on learning. How about asking why it is the way it is?”
> greatest failings I've seen in leadership in our time is the idea that in order to make a critique one must come with a solution in hand.
The pattern works at very high levels in an org chart, but with developers and those that manage them it breaks the whole concept of problem solving. You have to be able to identify and understand problems before you can come up with a solution... and usually, with software, the solution is developer hours.
I always pick apart previous teams' work.. it's how I learn. I question most every decision because I'm curious why they made those decisions. And it lets me think about how I'd do it better. And yes, I know that many poor decisions are not necessarily the developer's fault. It could be bad specs, lack of time, etc.
In most cases, "better" means different things in different contexts. (Customer-driven vs performance-driven, for example) Of course, this isn't news for most of us. Where I think a lot of us fall short is assuming that definition has changed since the code was written.
This harkens back to Chesterton’s Fence. It’s always worthwhile to interrogate why things were done the way they were, especially when first coming onto a project. Knowing the why of a decision is essential to understanding if and how it should be changed. Especially if the reason is “this is what we had the time and knowledge to do at the time.”
> It’s always worthwhile to interrogate why things were done the way they were
It really isn't. A lot of the time you end up spending a lot of effort to understand something that was dumb to start with and has been dumb ever since. Something like the bullshit asymmetry principle applies - any idiot can take 5 minutes to write a line of code that will baffle a team of experts for hours. (I've done so often enough myself).
The people who have those answers have long gone. The only person left is a project manager who tells you it's up to you to figure it out. After you make a change in production they will come to you with questions after a few months, just when you assumed things must have gone well
They usually shut up when they realise the "previous team" was one of the founders who is paying their wage.
When your v1 code takes a company to profitability it was good enough.
Oh man this one hits home. I don’t do much coding anymore but my general advice to folks I lead is you’re never going to be happy with how you did things and just make sure it scales and is well tested.
Edit: oh and how could I forget as simple and readable as possible
At this point I just don't think total rewrites from scratch are a good idea, full stop. I've never seen a rewrite from scratch that didn't lose most of the learned solutions from the previous attempt, repeat most of the same mistakes and have to re-discover the solutions, and utterly fail to even attempt a passable improvement on a model of the core complexity of the problem being solved.
I'm not granting a "rewrite from scratch in Rust" exception even though that's in vogue right now. I'm not saying don't rewrite it in Rust, I'm saying don't rewrite it from scratch. It's harder to write new features in Rust while maintaining the old C code, but it's the right way to do it.
Second System Syndrome! :)
https://en.m.wikipedia.org/wiki/Second-system_effect
AKA fighting the last war. Can be difficult not to focus on trying to avoid painful things from past projects, though.
And the only better experience than that is that all software is pain
So true!
I always tend to say: "Everybody learns better from their own mistakes, but if you are empathic you can also learn from other people's mistakes".
The latter is less costly and only requires you "only" to open your eyes and look at projects that are in an ugly state the right way. Yet surprisingly few people are capable of looking at someone elses fucked up project and not going all like: "Hah! Idiots! I would never have made that decision".
Maybe however that crusty piece of code used a framework that — back in the day — was the hottest, trendiest piece of technology out there and you are currently in the process of committing similar sins, and you won't know it till it is too late.
For me adminstration of Linux servers has been an invaluable source of inspiration. You are directly and 100 percent exposed to the effects of software aging in a changing environment. And you directly wittness which software ages like fine wine and which ages more like milk.
> which software ages like fine wine and which ages more like milk
Now I'm curious. What software has aged well? What software hasn't? Do certain types or categories of software tend to age better or worse?
cURL is my personal go-to when discussing things like this. The Unix implementation is old, stable, functional, and has changed elegantly with the times.
Especially when they're your decisions, decisions you pushed for and felt very good about at the time.
Yep!
There is no teacher quite like cause and effect.
Reinforcement learning, as usual, for the win!
This is literally the approach recommended in Team Topologies. The same team should own vN and vN+1, so that they both have to operate their own design and have the opportunity not to make the same mistakes again. It should be the default.
That would require staying in one company for more than 2 years, which means you probably lost out on 20-30% of income over next 2 years.
It’s okay to enjoy work and focus on your craft for two years rather than jump for more money. If you like your employer and colleagues and are growing as an engineer isn’t that better in the long term? I think jumping around can risk creating an engineer who leaves a place worse than when they started.
> If you like your employer and colleagues and are growing as an engineer isn’t that better in the long term?
No, growing as an engineer just means the reality of software development will make you more miserable.
> I think jumping around can risk creating an engineer who leaves a place worse than when they started.
Of course it does, but companies have made the choice to pay more for that than for someone who stays and makes the place better, we should give them what they want.
> It’s okay to enjoy work and focus on your craft for two years rather than jump for more money.
These aren't mutually exclusive.
> If you like your employer and colleagues and are growing as an engineer isn’t that better in the long term?
Better than liking your employer and colleagues and growing as an engineer and also having more money? No.
This isn't a field with low demand for talent. You can have all these things--don't settle for less.
> I think jumping around can risk creating an engineer who leaves a place worse than when they started.
That's a risk that companies should be willing to pay to avoid.
But a lot of companies aren't willing to given their engineers adequate pay increases each year, and it's not engineers' responsibility to accept less money than they are worth. If you want more experienced engineers, you have to pay for more experienced engineers. That's just basic free market economics: if you don't like that you don't like capitalism.
Some companies understand this and do better, but these are the exceptions not the rule. If you find such an exception, stick with them!
I fundamentally disagree with the narrative that engineers are supposed to be passionate about their craft and learning and interest and not care about the money. That's propaganda spread by corporations to get us to accept less pay, and if you believe it you'll be exploited. And as it turns out, the places that pay the best are also usually the best places to be passionate about your craft, so there's no real conflict. The places that pretend to pay you in learning and passion instead of money generally don't deliver on the learning and passion either.
It does not. It requires the team to be stable. It says nothing about the composition of that team.
In particular, I appreciate that you assign value to the consequences and not the decision itself. Anytime junior engineers on my team would complain about "shitty code" I'd assure them that someone would be complaining about their code in a few years.
Having the context or, better yet, responsibility for the past decisions is great for developing a pragmatic approach to software design AND empathy for other software engineers.
Came here to say this. Ive had to work on slop, and it never stops being slop unless the stars align and you can do a bunch of work which is short term unprofitable. What people really should do is work on the same project for five years. Watch the interest wax and wane and more importantly watch the requirements change. Then you'll stop looking at things in black and white like "good project bad project"
Too bad a lot of devs love to greenfield, slap it together, then bugger off to the next project pronto.
When that's both more pleasant and better paying, who can blame them?
"You either die a hero or you live long enough to become the villain"
Never thought of that in the context of software projects. It works really well!
oh the hills that were conquered and those to have died upon, captain hindsight strikes again!
Come on to a brownfield project and ask a lot of pointed questions about how we got here.
You're also not a senior engineer until you've been forced to adopt the coding style and practices of that legacy project. A lot of younger developers like to come into a new project and slap on the latest frameworks or coding practices that they've picked up recently, without acknowledging that breaking the standards of the project lead to higher mental load to maintain that code over the long term.
It isn’t just younger developers doing this, nor even inexperienced ones. I’ve seen a ton of experienced people do this to themselves and their teams, mostly it seems to be able to pad their resume, but some honestly believe newest and shiniest is best.
If someone has only used a handheld wood sander, they will be ecstatic when someone shows them a powered belt sander.
But someone who has used a palm sander, orbital sander, belt sander, disc sander, and so on will be able to see a new type of sander in the context of existing tools and know what’s right for what job.
Your average junior will be in the first scenario. However, it’s possible to be experienced yet still be stuck in the first scenario.
Juniors with tons of experience aren't very uncommon.
The way i've seen it is Junior/Senior as a label of wisdom/engineering acumen and experience (not just a title) is not directly correlated to years of experience. Some people just have the same year of experience/level repeated 3-5 years. Some "Junior" with 1 YoE can come away with learnings some seniors never get in their careers. Not wholly their own fault, but it's a fact.
I cannot grok this. Are you saying that “juniors with tons of experience are very common?”
Do you mean that there are seniors with a lack of experience? I believe yes I was also making that same point.
Ageism in the industry also encourages seniors to appear to be plugged in with the newest shiniest thing, so we kind of do it to ourselves with our hiring practices.
We need to divorce seniority from talent.
As the adage goes: Some senior developers have 20 years of experience, and some have one year repeated 20 times.
“Juniors with tons of experience are common” means that people with many years of experience who are nonetheless at the level of a junior engineer are common.
This phrase is divorcing seniority from talent, albeit perhaps not in the way experienced engineers might prefer.
Seniority from skill, most likely, talent wouldn't change over time
> A lot of younger developers like to come into a new project and slap on the latest frameworks or coding practices that they've picked up recently, without acknowledging that breaking the standards of the project lead to higher mental load to maintain that code over the long term.
This is an organizational problem.
Right. At my new job, the framework-level choices come from people like 4 levels above me. Problematic for a lot of reasons, but juniors slapping frameworks onto things is not one of them.
I’m actually kind of struggling to imagine a scenario where a junior developer has the agency to slap entire frameworks onto existing systems. Maybe a late stage startup? Coding practices, definitely.
Enterprise.
We are currently going under modernisation converting Perl to Python. It's what it is, but I've encountered developers coming along and wanting to throw bulky frameworks at when all you require is to process the output result of a vendor issued binary.
There have been novel ideas but just unpractical for the causes.
It's madness getting through to them when all you require is a replica of the current system and not a rewritten feature set not related to the scripts at hand.
Coding practices are another tool with trade-offs involved in their use. Mental load is sometimes another word for learning. So I like to hear it from juniors because whatever is ultimately decided, it's an opportunity for them to have a real discussion around trade-offs and the benefits versus the costs of a particular approach which is invaluable for them.
> A lot of younger developers like to come into a new project and slap on the latest frameworks or coding practices that they've picked up recently, without acknowledging that breaking the standards of the project lead to higher mental load to maintain that code over the long term.
Using the latest framework improves their future job prospects. Whatever company they work for will lay them off in a split second and won't provide anywhere near the comp increases that a new job would. So why should they care about the long term of a project?
Companies have no loyalty to employees which means employees have no loyalty to companies.
> So why should they care about the long term of a project?
Because your future networking potential—and therefore future job prospects—depends on being liked by your peers, who may not choose the same moment of departure as you would. Follow standards and work towards long-term maintainability not because you see yourself at the same company in 2 years but because your coworkers might see themselves there and you want them to like you and feel good about recommending you to their network.
The co-workers that matter the most for networking are the ones playing the same game. They will move to new companies more quickly and try to rise up the ranks more quickly. They will know how to write a good referral because they want to maximize their chances of referral bonuses. They will have the ear of the EMs and Directors because they know that maximizes their own career options.
Saying you're right about which referrals matter (which I don't accept, but I'll play along): you're presuming that not only are these people playing the same game but they're willing to help along someone else who they know has no interests but their own at heart.
Why?
Why would they want to help you along? They burn capital referring you, they burn even more when you make a mess of things. When you leave a burning husk behind everywhere you go your own network of people who trust you is minimal, so you're not an asset to have around them. Why would they want to help you instead of finding people to con who can actually be an asset to them? In this entirely self-centered world you've put together, what's in it for them?
> you're presuming that not only are these people playing the same game but they're willing to help along someone else who they know has no interests but their own at heart.
Referring people for no other reason than that you know them works great, since it will implicitly happen in reverse too. See e.g. the Freemasons.
> They burn capital referring you? They burn capital referring you, they burn even more when you make a mess of things.
They burn capital if hiring you seems like a failure, they win if it seems like a success. That means your incentives are aligned. No-one gets dinged for hiring someone who implemented a couple of buzzwords, left after a couple of years for a higher position elsewhere (or got promoted into management), and then the project they were on failed a few years further down the line for unrelated reasons. Quite the opposite.
Personal pride? I flat out don't respect a developer who refuses to do anything right because of nebulous hatred of any and all employers. No they don't care about you, no that isn't a good reason to go full apathetic.
It's not hatred but indifference towards the employer. That indifference is mutual. I'm not saying that I agree with it, but I get it.
It's not hatred. It's self-interest. The very heart of capitalism. A rational response to incentive structures.
You're the one sacrificing your own self-interest for pride and are upset that others refuse to do so.
edit: And I get it, I don't optimize my own career like this, focusing more on what I enjoy to work on at the moment. But I don't begrudge people who do optimize nor do I claim to be better than them. If anything they are acting in a much more rational and organized way to their goals than I am.
"More rational" is subjective. There are values and incentives, other than money and specific kinds of self-interest, that can bring satisfaction and joy.
And why should someone be respected if they are intentionally making others lives more difficult like that. That's just hypocritical.
Moving to a new framework that will look good on everyone's CV isn't making people's lives more difficult, quite the opposite.
There's a tradeoff here too. If your codebase was started in Java 6, I dearly hope you won't continue to write code in that style anymore for new code.
How about Fortran 77"
Six-foot-four-tran? (I’m sorry.)
haha damn it I didn't see that typo.
Not to gatekeep, but I question whether this is even a legacy project, or just "not greenfield."
Legacy projects are usually have shoestring budgets for any changes. After all, they are legacy for a reason: likely nobody has touched the system in a while. I can't think of a single legacy client I have that would be okay with a company wasting their hours on adding a new framework. It can be hard enough to convince them to let us add tests or upgrade from PHP 5.6
It's not limited to shiny new frameworks. Following naming conventions, design choices, etc, can take work, especially if you see something that isn't "best practice". Heck, sometimes you need to refactor something and the best thing you can do is reimplement any bugs you found (obviously with documentation and discussion to try to surface the why)
I used to work for a company where the American devs indented with 4 spaces, and the German devs indented with 2 spaces. When they finally decided to standardize the code style across the company, they couldn't agree on indentation. So, they compromised and everyone started indenting with 3 spaces. I swear I'm not making this up. I worked on one codebase that lived through that transition, and some source files would have all 3 indentation styles scattered throughout.
Probably the best argument for tabs I've seen so far.
> breaking the standards of the project lead to higher mental load to maintain that code over the long term.
Nah. Over the long term the lava layer pattern is the only way to stop the mental load of the project from growing indefinitely. You will ultimately have to keep up with modern development practices, because keeping everything on standards from 2010 will ultimately be even worse, and the longer you leave your migrations the more painful they will be.
As long as all the layers are actually being replaced. A bit of each pattern in each layer makes for a mess.
You don't and can't fully replace each layer, at least not in order - you should expect to have three or four ways of doing things in the codebase, that's just the reality of large old codebases. It still beats having your whole codebase be stuck on the oldest of those three or four ways.
> forced to adopt the coding style and practices of that legacy project
Let's keep going with this SQL-injection prone legacy framework, with zero automated tests for the entire codebase!
I've hit that, for real. Sometimes you really do need to move mountains to change legacy.
"checkout project, reformat all source files, commit"
I've had innocents' IDEs do this to them. Fortunately it results in pull request rejected, not commit to master.
lol I love when people do this and it and now whenever I `git blame` their name shows up.
Agreed, this is quite annoying, but there is a workaround. You can tell blame to ignore commits like this, through a config option blame.ignoreRevsFile or similar CLI option. Not the most convenient perhaps, but it’s something. I believe GitHub also supports this, though you may have to request it to be enabled in your repository.
Given this, I tend to prefer a single, formatting-only commit when introducing formatting standards to an existing codebase. Otherwise, it’s difficult to take advantage of QOL features like auto formatting in your editor, or other formatting tools which tend to operate on entire files. Then PRs end up being mixed with formatting changes, which adds friction to the review process.
FWIW, git now has options to ignore such commits. In a big codebase each big codemod sha should be added to the blame ignore file .git-blame-ignore-revs. It's supported by GitHub out of the box
https://www.stefanjudis.com/today-i-learned/how-to-exclude-c...
Sure. Of course you can't commit to the main branch and your commit will never pass code review (which is a requirement for merging), let alone the automated tests in CI that apply formatting and fail if anything is not formatted as it should be.
These safeguards save so much time that there should be no excuse for not having them.
Not wanting pipelines to fail because someone put an extra space after the line, or because someone's text editor didn't put a new line at EOF, or put a unicode BOM or whatever is a pretty good reason not to have this "safeguard".
Just to be clear, the argument for this has nothing to do with how the code looks and has everything to do with how you can swallow a pill, once, and in exchange you get clean rebases forever. You can hide the commit from blame consideration.
I've seen changes like this drag out over a year because the product had so many permutations that thoroughly testing any change was an exercise in futility. I've also seen changes like this fail to happen just due to a lack of cultural desire to make things happen.
The first one is more forgivable - sometimes that's just the problem space. But neither of them are conducive to fulfilling working environments.
Might be true, but as someone who worked on a few legacy projects, the opposite, i.e. "You're not a senior engineer until you've worked on a greenfield project" is also true. There are skills that you won't pickup by working on old piles of code. Knowing what technology to pick, how to design your system for high-availability and how to avoid needless complexity are just as important to a well-rounded senior.
Perhaps in the end you are not a senior engineer until you've done a bit of everything.
Personally reading this thread I just realized that devs encompass so many areas. Requirement for senior engineers working on practical formal methods, real-time systems, Linux-kernel hacking, or those writing CUDA kernels all day are all gonna be different. System design is only for a broad kind, but not for all kinds, of senior engineers.
I agree with you. But, I have found the middle out effect in online discourse means we get articles (such as this one) polarising viewpoints for attention. Sadly, it works.
There's no hurt feelings like the hurt feelings of a junior engineer, who has spent the last year kvetching about how much they hate working on legacy junk, hearing someone else refer to one of THEIR projects as "legacy junk".
Any code that's old enough to have its first birthday party is "legacy", which means that "legacy" is a completely useless category. Anyone calling anything "legacy" is generally just showing their own lack of experience.
1 year old code is not legacy. Legacy is a useful category. COBOL is legacy. Maybe it is unclear exactly where to draw the line but if that were a valid reason to discard conceptual categories we wouldn't have any.
I like the definition from "Working Effectively with Legacy Code", Michael Feathers says legacy code is code without tests.
Heh, so true in many cases. I had rewritten a utility that scanned a directory and moved files to s3 from Perl to Go and eventually a different team took over the code. A bug popped up and they were not confident to update the "legacy code." I could do not but chuckle to myself: this was like a fee hundred lines of well organized and fully unit and integration testable code with a readme and runbook and grafana/prometheus and structured logs aggregated in Splunk. And that has been running for like 2 years. They just didn't want to even attempt the fix and in fact pushed it off indefinitely.
Legacy code is code without tests. Doesn't matter if it's new or old.
IMO legacy code is code that has lots of if statements and special cases sprinkled in over the years to enable new features quickly and get out immediate hotfixes for bugs without doing full refactors and/or data migrations to address the root causes. Even with 100% test coverage it’s a pain to build new features in because there are so many paths to think through, and instead of single sources of truth each part of the app assumes every other part is working in very specific ways.
An alternative definition, legacy code is any code where there’s no one left on the team who has been working on it for years and intuitively knows the pitfalls. Then everyone’s scared to touch it or make big refactors, which actually leads to those small special cases being added instead.
It's funny because 90% of my experience has been legacy projects, and the greenfield projects for me were the hardest, because it's really hard to organize as a team to build something from ground up, not kill each other with shovels in the process, and not introduce technical debt early on that will be impossible to pay off down the road.
When joining a legacy project, most of the hardest work has already been done for you.
Also, with legacy projects there is usually an array of helper scripts, classes and functions that do a lot of lifting. You have to rebuild all those as well.
Did those teams have lots of senior engineers? I’d assume when you have a single tech lead over each area the team won’t fight each other since junior engineers will just follow their lead.
There's also much more technical pressure on a greenfield project mainly because if it turns bad it's "your fault". There's no legacy to blame, no excusing bad choices (you made them!). It is part of the responsability but if and when it goes sour there's a bigger feeling of impostor syndrome than on a legacy project.
I started my career working on a large legacy project. It was the best learning experience, and super interesting. It certainly did not make me hate legacy projects. Work was like an investigation into who did this weird thing and when and for what reason - and there was usually a good reason. Then detangling if that reason is still valid today, or if we have time to refactor the thing appropriately to rewrite it in a different/better way, or how we can solve this obscure bug without breaking five other usages, etc. And doing all that as elegantly and non-disruptively as possible with hundreds of people on a tight timeline relying on the codebase. It was an extremely satisfying challenge.
I unironically like working on legacy projects. It's good to know it stood the test of time and my efforts spent on it won't be wasted, because it is unlikely to be dropped.
Of course this assumes there is a way to introduce improvements at all. Just keeping the lights on isn't as satisfying.
I love it too! So much to see / do / improve
I enjoy it too, I explained why in an SE answer some years ago and I still feel this way - https://workplace.stackexchange.com/a/139878/5229
Legacy or not, I don't consider anyone really senior unless they have spent building and more importantly maintaining/fixing a system at least for 3-5 years at one single place and have done this at least 2-3 times across different companies. We all know that it is a lot easier to start a new/greenfield project with shiny tech etc. It is a lot harder to maintain especially once it is 2-3 years old and in production with tons of users.
The challenge there is that "legacy" codebase are just so much slower; you spend more time figuring out what needs to be done and how, than just writing code like in a greenfield project. And everyone knows the latter is just so much more fun.
No a senior engineer is someone with at least 3 years experience, haven't you seen any job ads?!
It's like credit cards. You start with senior and from there it's gold, platinum, diamond and so on.
That's nothing, I don't consider anyone a real senior unless they've maintained something for 10 years!
But seriously, everyone tries to move these goal posts. We still have no good idea.
I've run into a few engineers, very capable people and I'd be happy to work with them, but their experience was limited to very structured environments. It sounded like they were in well organized organizations.
One in particular worked for a sort of "side ways competitor". They were horrified to hear about my deploying a fix all by myself, no review, for a legacy product that really didn't have a good testing environment. It was inconceivable to them.
On the other hand, not long after, our organization of 4 coders put out a better product in 3 months than their massive organization did in 18 ... and using that some "inferior" legacy systems that just so happened to be better suited to the job than some fancy new framework.
> deploying a fix all by myself, no review, for a legacy product that really didn't have a good testing environment.
This sounds like a great way to cause a live-site incident.
It has risks, no doubt about it.
It's not one or the other though - a bit of safeguards here and there would make you even more productive, imho. Code reviews are indispensable for sharing knowledge within a team, automated deployments help deliver the system reliably and without unplanned downtime, and tests... People start appreciating tests when the red lights warn them about breakages in some completely unrelated places that they didn't even think were affected by what they did. And I'm not talking about 100% coverage - even covering just some of the trickier, but important flows helps immensely.
I don't disagree generally, and in fact we're starting to build those safeguards where I am ;)
Having said that process and bureaucracy is still what they are and they tend to eat up more cycles as time goes on, even if only because they're administered by humans.
Deploying straight to production? Eh, we've all been there. It happens.
No review though? Aaaa
Legacy projects are great! If you don't have any current critical time pressures, you've now got yourself a beautiful puzzle to slowly unfold and understand. If you've got things like git blame annotations turned on you can read commit messages, see past pull requests and really start to get a feel for the people who worked on the project previously and understand how their minds work.
The biggest challenge is of course trying to prevent those feelings becoming contempt, rage and resentment as you see the same mistake repeated over and over again by one person who should know better while they make funny little comments in their commit messages while writing bugs that end up causing data corruption a year down the line.
The trick is mindfulness meditation and definitely not going to LinkedIn, printing out a copy of their face and then ripping it up while screaming.
I know a guy who always says you're not senior until you've had to maintain your own project that's become legacy and I agree. Too many developers serially job-hopping every 18-24 months. They all have experience with other people's legacy code but not their own. They're all happy to build something new but if they don't stick around long enough to see the consequences of their decisions, they're lacking critical experience.
I job hop a bunch and a common theme that I have had in my jobs is to go in and understand the legacy, and bring it up to a more reasonable state. It's been interesting having to be the one dealing with someone else's consequences.
I'll sometimes chat with the people who have had to maintain whatever ground work I had laid, and each time around, I get a little bit better at laying down a framework of infrastructure and operations that's better and better at staying sustainable
By this logic I've been a senior engineer since I started working!
Senior engineers need a better grasp of boolean logic than this.
You’re not a senior engineer until you understand the difference between necessity and sufficiency.
Refining/redesigning a process can be way more satisfying in the long run. Just have to remember to document those "avoided writing code" solutions for your annual review or to add to your resume.
Same here. My first job was “take this spaghetti mess and make it better”. It was an old crusty Perl CGI codebase without any framework and a ton of bad decisions.
Necessary, but not sufficient.
By your reasoning, looks like you are still a junior /s
s/\/s//
Junior engineer loves the rush of figuring things out.
Senior engineer has come to terms with feeling like an idiot all the time and now seeks this feeling because that's where the progress lies.
Nothing else to it.
And I would add, you are not an "expert" until you had to reverse engineer a program who's source disappeared a decade or more ago :)
But an interesting article for modern times.
Or the logic created by other staff that's long gone.
"This is wrong!"
Well, we actually do it that way because Frank told us to do it that way
"Frank isn't even here anymore! That doesn't make sense! Let's just do it the right way."
Three Weeks Later
This is usually doable until it's DEWS/SAGE radar code from the 50's. That's what NASA started with and the last guy died in the 90s. We got every other subsystem off the mainframe, but had to keep it around for the radar stuff until 2004 or so.
I sense an interesting story.
Details please ;-)
https://www.baen.com/rip_moc
1990: https://www.ibiblio.org/apollo/Documents/MCC%20Development%2...
Pirates: https://www.wbs.ac.uk/news/the-nasa-pirates-that-transformed...
Head Pirate: https://historycollection.jsc.nasa.gov/JSCHistoryPortal/hist...
WEX/Display builder/etc: Even when I was there my young peers weren't too keen on legacy/back-end understanding. GUIs were the cool thing. I worked on something that touched many legacy subsystems so got to work with a lot of "dinosaurs" (what the Pirates called them).
MCC testing was all AM shift, pirates/youts up front, retired controllers in the back. Test MOC took 30 minutes to recycle. I always sat in back if they'd let me.
In contrast to sibling comment: this is actually quite fun in small doses. Wouldn't like to have to do it for an entire product, but I've done some "competitive analysis" before.
I had to do this once and, while I do recommend the experience for professional growth, it was not fun for the most part!
I don't like gatekeeping, but I think a Sr should be a well - rounded developer. They need to have seen enough issues to be able to make solid engineering decisions.
Every project either fails or turns into a legacy project.
I would have loved more detail in the "The limited role of developers" (or maybe merits a separate follow up). On the one hand, this section feels so obviously right. On the other, roles like ops and QA evolved for reasons so curious in wrangling with this legacy product how that could've been better / different beyond just consolidating work / roles.
"The only thing left was to deploy it so that we could test it. This was a problem, of course – in this case, we were not the ones deploying it or testing it. Deploying had to be done by the ops team, and testing by the test team. Why couldn’t we, as developers, manage the whole feature from start to production instead of opening tickets and waiting for other teams before we could close our task?
Firstly, we couldn’t avoid manual testing since much of the code wasn’t covered with tests. We also couldn’t deploy the application ourselves since the infrastructure didn’t allow us to.
This had us thinking about the reasons behind the separation of duties and how our current approach is better. Seeing that our lead and cycle time on this project was significantly higher than usual (it took us weeks to deliver something we usually deliver in several days), the evidence was strongly in favor of that."
"with almost a decade of experience" isn't a senior at all with or without legacy projects :)
and nothing beats the legacyness of languages before the java/webtech times - and im not even talking about COBOL
Not even COBOL, eh! https://www.youtube.com/watch?v=3d1phCytJyg
One of my most educational personal project has been to update a ten y/o mupen64plus OpenGL plugin (Rice) to OpenGLES for RPi.
So much code that looks weird that I had dig in a lot and finally understood why this tiny piece of non-sense code is actually what make the whole system working, and think about how smart the previous dev was and how dumb I would have solve the problem myself.
I prefer a flat structure where you're a junior developer if you're still in training. I've held positions where my title was senior while I worked on a few projects, both legacy and otherwise. However, I think labels like these are harmful and create an atmosphere of authority and gatekeeping. I still learn new and exciting things from everyone, including junior associates. On the other hand, I know people who label themselves as senior who could be called junior.
After enough time, none of these HR provided labels matter and the work you do signals your level, not the other way around. Just a fair bit of warning though, regardless of your level, you should know how to implement the solutions from the ground up without corporate support and IT setting everything up for you.
For readers, programmers, and engineers stopping in here asking "ok what can I do to better myself here?"
I really recommend "Working Effectively with Legacy Code" By Michael C. Feathers. It is a top notch read, and helps newcomers (and vets alike) approach these legacy systems.
Marianne Bellotti’s “Burn It With Fire: Manage Aging Computer Systems (and Future Proof Modern Ones)” is another good book about working with legacy systems, though its focus is more about managing system evolution and modernization than specific code techniques.
Adding that to my list thank you very much.
A company I worked for had that classic problem of there being various older systems that didn't "belong" to any team and which nobody was confident in fixing or changing.
They did something which I thought was brilliant: they gathered a list of ALL of those projects and assigned every single one of them to an existing engineering team, whether or not that team had any relevant knowledge or experience (which was true for most of these older projects).
No team could complain about this because every other team was suffering through a similar challenge.
I think it worked really well! Bugs in older systems now had obvious owners, and teams were incentivized to dig in and figure out how the crusty old code they were now responsible for worked.
Projects are like people. Everyone shows great interest when they are being planned and brought to life. Some happen by accident and are more or less unplanned. Initially, everyone is enthusiastic about the new baby—or project. In the beginning, a lot happens, and things take unexpected turns. But after some time, things start to settle, and people's interest fades. Only a few loyal friends remain. Eventually, the project becomes a burden, and very few are willing to take care of it. In the end, many just want to let it rest in peace.
I find it incredibly satisfying to slowly figure out, document, tidy up, and very gradually morph a legacy project with the puzzle-like constraint of not actually breaking stuff or screwing over legacy consumers.
I think it’s less awful if your expectations are not that of a new project. Yes, there’s a ton of ugly bits at the interface level that you can’t fix. You make it as comfortable as possible while figuring out where to go next if anywhere.
We were lucky enough to experience explosive growth of tech industry in the past 20 years. A perk of this golden age is that one did not have to work on legacy code at all, if so we chose. Better, we got to build 0-to-1 systems[1] that could make a difference, like the first search engine, the first key-value store, the first machine learning pipeline, the first resource orchestrator, the first data pipeline, the first real-time OLAP system, the first distributed logging system, the first bespoke compiler, and the list went on.
Unfortunately, we had this level of luxury because we had growth, especially new business scenarios that involved thousands of companies and millions and millions of users, thanks to the confluence of internet, mobile, social network, cloud, and machine learning applications. I'm not sure we are going to have similar growth in the coming 10 years, but I hope I'm dead wrong.
[1] The downside, of course, is that engineers tended to abandon a project after version 0.1, like what Twitter engineers did to their Manhattan project.
You're not a senior engineer until you get the skill to turn any little thing you do into a one-hour presentation and a chapter in a book.
This is too real
Or a 6 paragraph Slack post in #general about what an amazing experience it was...to host a 30 minute Q and A on a certain topic. #community #growth #sharingiscaring
Well, if you are long enough in the industry, in retrospective, most of the code, frameworks and platform you have worked on are now considered "legacy", given the dynamic fast pace of our environments.
Data-centers -> Cloud Monolith -> API and Services Databases.... Memcache -> Redis JS/JQuery -> MVP Framework Git. and not to forget K8S.
and this is just the tip of the iceberg.
Is there a significant amount of people not working on code that already exists? That's like the total opposite of my experience
"Code that already exists" isn't necessarily the same as a "legacy project", though everyone's definition of what "legacy" means is different.
I guess, how do you define legacy then?
Old tech
Honestly, it's a "know it when you see it" situation mostly, but there are some strong smells indicating there's a legacy project in the walls:
- none of the original authors are with the company anymore.
- it's written in a language/using a stack/using technology that is used absolutely nowhere else in the company anymore.
- it is big enough that migrating all of its functionality is a tremendously difficult undertaking, to the point where it's likely never going to get done, at least not in your lifetime (either in terms of your tenure at the company, or literally, before you die.)
Outdated architecture/code practices/patterns are a smaller smell, as is the lack of documentation/testing - but those can be present even in greenfield projects written by a big enough dipshit - and I say this having been that dipshit at several points in my career :)
In one of my past job, for some reasons I was the only one diving into legacy 12+ years old code.
The other members kept producing new code and debugging new code.
Disregarding of whether it's fair or not, there was a kind of balance that kept the boat floating.
I left since then and it wouldn't surprise me if the legacy was being rewritten from scratch (and over-engineered).
Legacy 12 year old? Try 40 - 45 year old code.
Back when people mandated an 80 character line width because their CRT monitors only had so much resolution
IMHO 80 char lines are easier on my eyes, independent of monitor tech. I get antsy all the way over there on the right! Past 100 chars I am simply less confident in being able to pull off the ocular carriage return to the next line.
I find 80 characters to be too short, and the resulting code after passing through a linter is significantly harder to parse due to the indentation being all over the place.
I always found that even at startups we had to at least integrate with legacy solutions.
Most engineers will scoff at the idea of patching up a legacy project if they see the slightest deviation from “best practices”. They will slam their fists on the table and claim that management keeps “piling on tech debt”. They will argue for a total rewrite and dismiss any concern of said rewrite taking years because “this is what it means to have high standards and best engineering practices”. They wear this as a badge of honor and frame the conversation as a question of morality and purity, in which they of course have the upper hand since they are not motivated by petty business concerns such as profit.
Engineers that refuse to acknowledge constraints, whatever the nature of those constraints may be, are not engineers. At best they are ideologues, at worst they are just incompetent. The most pathetic thing you can do is just continuously deny the laws of physics and reality, because it doesn’t suit you at some ideological level.
Truly elegant solutions are those that account for all constraints in the simplest, most concise way. It is those that do more with less.
They will argue for a total rewrite
in my almost 3 decades in this industry THIS has been the number one way to tell a Junior from Senior SWE. nothing else comes close 2nd
I usually say that when people say “legacy project”, it’s really “code I either don’t understand or don’t like”.
I specifically mean code not covered by automated tests. And this almost always falls in the dislike/misunderstand pile for me, but not conversely.
I'd argue that even with automated tests it can be considered legacy. Easier to change things if the test coverage is comprehensive enough, but I've yet to find a project like that, even the ones I built myself.
I hope you find one one day. It makes such a difference to have a test suite which you trust, and which doesn't erupt in false failures at the drop of a hat. I completely agree, not all tests are created equal.
It is interesting, if there is a bug reported in the legacy system, the ticket can bounce between juniors and mids for months without fix in sight. It's like they don't have ability to read and debug code.
When it is something new, it is "easy" - check Stackoverflow or ask LLM and then modify code until it "works" - which doesn't even need any understanding of how it works. Just tweak until there are no errors and screen makes the manager happy.
Since corporations started cutting budgets - don't pay for training and get developers who know how to write CV and memorised interview questions and agree to work for substandard pay, it is all slowly going downhill. More experienced developers are tired of handholding new employees through basics or very much developing the tickets for them on top of their own workload and so they quit and problems are getting bigger and bigger.
It's a very interesting experience working in a large codebase that's a few years old while we're in the middle of trying to dramatically rearchitect everything because we realized the old way of doing things was creating tons of technical debt and confusing abstraction. Even worse, we're always trying to optimize things because we've got some soft real-time requirements and we're using about 50% of our entire time budget after we upgraded our bare metal servers to Zen4 (it was more like 70% before that). Every fix turns something else into the long pole that needs to be rebuilt with the lessons we've learned about writing clear, concise, high performance code.
As it turns out, if you're operating on large data in bulk you should probably just have a dense ID map and then do everything in arrays. Nothing beats arrays :)
I kind of love working on a legacy project, its like reading a love letter from past devs, their hopes and dreams, classes describing an abstraction that was never used. It's like reading hate mail from past devs, towards their bosses, towards ways of doing things they didn't like. It's like uncovering a stack of newspapers from a time before agile, when people had eldritch and archaic ways of naming tables and variables and methods, when they spoke of things we've forgotten.
Sometimes it is a pleasure and sometimes it feels distant and alien and frustrating.
Only a little senior, if you're still working and not sitting in meetings all day - rungs remain
Why is sitting in meetings all day considered the top of the IC developer ladder? I don’t want my career to end up like that
Beats me, I didn't want it either :(
Counterpoint: You're not a senior engineer until a project you've started has become a legacy project.
Nothing better than somebody exclaiming, "what the hell were they thinking when they did this 20 years ago!" and the "they" is you.
There are two axes:
-------------------
Axis one: quality of the code
Axis two: how many use the code
There are four quadrants:
-------------------------
Clean code no users: enjoy it while it lasts, it won't.
Clean code and users: what are you doing here, the code is done. Get out!
Dirty code no users: a waste of time
Dirty code and users: careers are made here.
Everybody hates working on legacy projects, myself included.
I am proud that I am a part of a team that uses most of the best practices
Speak for yourself... I'm the exact opposite. Then again, outside of work I fix and maintain other physical systems which are many decades old, some close to a century, so perhaps that changes the perspective. I see so many in the software industry cargo-culting "best practices" dogmatically, creating constant churn via useless trendchasing, that they keep reinventing - badly - new ways to do the exact same things as before.
> I see so many in the software industry cargo-culting "best practices" dogmatically, creating constant churn via useless trendchasing, that they keep reinventing - badly - new ways to do the exact same things as before.
That's why software engineering is still very immature as a science field. It's pretty much the result of so widespread software engineers' inability to even define the problem they're trying to solve. Along with the inability to demonstrate the impact of the change and provide at least some cost-benefit analysis.
And this is spreading like a cancer. Unfortunately, I see this almost everywhere and I am not sure when it all began because we were certainly not being taught such things at the University. Critical thinking seems to be lost in a signal-noise ratio.
Then I became a senior engineer pretty early - as my first job was working on nuclear power software. We had code in Fortran that NOBODY was allowed to edit. But lots of other legacy Fortran was up for grabs.
Oh, and my favorite software tale is when we turned on the new software in the control room and the first thing the plant manager said when he looked at the new digital instrumentation was "that can't be right".
Refactoring legacy project while keeping the business running well is an arts.
A doctor who can cure patient illness from mallpractice are considered senior !.
Regardless their years of experience.
I really like turning legacy code into non-legacy code. It feels like getting to solve a mystery.
For example, maybe there's really crazy hard to follow code and the person who wrote it left a decade ago and there's no tests so it's too risky to refactor it but you can write new tests, really delve into the use case of the code and then refactor the original code.
Or, maybe the code is using a version of a framework that's 9 years out of date but you can incrementally upgrade it after gaining confidence in any way you can.
And you're not a _seasoned_ engineer until you've worked on a legacy project with declining budgets, declining staff, and ever INcreasing demands.
i've worked on many legacy code bases and i think an underrated skill that you need is courage.
courage to be able to push changes you think could be breaking changes but always be optimistic that you to be able to fix as soon as possible. tests are ideal, but not everyone has 100% coverage. sometimes you just have to trust your gut and do diligence and push the change. don't be scared to push something because you don't know the behavior in production. the only way to know is to try.
Agree. But you need confidence on being able to debug / log / fix / revert, so having a strong knowledge of some of the parts is mandatory (i.e. server if it's web-dev, etc).
Strong agreement from me.
I'd also add "until you've spent some time testing a code base you didn't write" to the mix. That definitely tightens your senses and busts some assumptions.
The next time you hear the phrase "... well, it worked on my machine ..." it takes on a very different complexion.
My first "software engineering" scale project was working on a existing massive 10 y.o js pub/sub and java templated webapp. Taught me why typed languages are valuable, why I should never listen to PMs, why I should be humble, why estimates are impossible but sort of important, and why long feedback loops anywhere in the entire process (including compilation) are productivity killers, along with not having an isolated private work area.
Learning to never listen to PMs sounds like the opposite of being humble
I was being a bit facetious with that one. In retrospect, I would have pushed back quite a lot more, but for different reasons than I did when I was in my early twenties. PMs are there for a reason, but the realistic pace that a given change can be expected to be made on this type of software, especially as a new team member, is a lot slower than I should have tried to achieve and that they would have liked, and it's important to avoid burnout.
I thought I needed to earn my place and tried very hard, but slowing down and going home at the right time regardless of whether the thing I was working on was done or not, would have been smart. They placed a lot of pressure on me to produce perfect code at the same rate as all the java devs based on the assumption the assumption their job was harder because it was backend, but the tools and parameters of their work were wildly better at the time, and I had undiagnosed ADHD.
So, I'd be protecting my sanity more and telling them it'll get done next week instead of trying to fit one more thing in because "it's just a button" or whatever.
All projects I have done for the first ten or so years of my commercial work have been picking up an ancient code base long after the original team has left, and making it work. I reserved the nice and clean code for my personal hobby projects. At work, it was about getting the shit from the past working and maybe improve it a little along the way. You even adopt the coding style you hate, just to minimize diffs and making it easier to navigate the history.
also, you are not a senior engineer until you’ve worked long enough at the same job to deal with the consequences of your own past decisions.
Great post! But I’d take it a step further:
> My feeling about legacy projects hasn’t changed – I still hate them (...) Instead of feeling resigned, we saw it as a place to ask questions and learn.
I see that you advocate embracing the opportunity to work on legacy projects, so I say even if you dislike legacy projects, try to avoid phrasing it so strongly. It might influence other devs in your team to develop a negative mindset toward them.
You are not a senior engineer until you work on a project from the beginning, leave no documentation, then leave the firm just as new assets are ramped up :)
Augmenting a legacy government / mission-critical app with a chrome extension was the highlight front-end experience of my career.
This may be true, but taking on too much responsibility in a project or having to do those responsibilities will make you a senior developer. Otherwise, it is not possible to get ahead of the senior developer you are affiliated with.
I could not agree more.
A big problem I see in software is that developers don't stick around on projects long enough to see that their mistakes are mistakes. Instead, they move on to the next greenfield project or rewrite the project in the newest shiny tech without ever actually addressing the core complexity of the problem they're trying to solve.
Ditto Network Engineer who hasn't worked on an ancient ISP, ditto System Engineer who hasn't managed an environment with decades old hardware and software.
If you can open up vendor documentation and the default solution works in your environment you are either lucky, or have gone through hell to get it there.
And didn't cry like a baby the whole time about every trivial inconvenience and absense of magic.
I was born a senior engineer then :))))) for more than half of my career I worked on legacy stuff and now I'm at a startup and I'm killing it.
All I have been doing for the past 10 years is deal with legacy. At least with the current one, the legacy is more recent and provides opportunities for myself and the other on my team to iterate on whatever we got, particularly in the realm of making operational improvements.
> decade of experience in pursuing clean code
Excellent.
So do I qualify if almost every project that I created in my > 50 year career is now legacy?
(Might be made slightly easier as many of the companies no longer exist.)
I still remember my first job porting a VB6 code to VB.net. The code was written before object-oriented code was a thing. Back then I thought OO would have solved it. Looking back all it was really missing was a strong type system.
I have a love / hate relationship with legacy projects. They're a fun puzzle, but often times the client has suddenly realized it's not possible to reach X and then I'm called in to make X possible. It can burn me out a bit.
I would also say: you are not a senior engineer until you’ve worked on a repo with the next kool aid tech where the repo’s owners left the company.
Just because it's the latest kool aid tech, doesn't mean it's legacy; it often feels like it becomes legacy as the people pushing it leave.
But this is why technological choices need to be taken with care, and if you're in a managerial position, you should only agree on using new tech if there's dedication / guarantees for the ones pushing it, a healthy labour market, or budget to provide training for people that don't know it yet.
And that's why so much is still 'boring' languages like Java, .NET, PHP and Javascript instead of Rust or what have you.
I'm a Go fan though and will exclude that, as IMO Go is by design easy to learn by anyone.
A senior engineer is senior because of his ability to solve problems (that juniors created). Legacy projects are a lot like a suitcase of problems that you never anticipated.
You're not a... engineer, engineer if you haven't worked on a legacy project. You're a junior engineer. Title inflation.
Truth.
I'd be happy to have a look at anyone's legacy code to at least write an analysis paper on it. (I'm currently seeking contract work, been doing code for a long time.)
My first job was working on a legacy project.
I could work as fast as I deemed fit, because nobody understood what I was doing. Any feature could take a day or a month.
Good times.
I found doing a vb to vb.net upgrade of quite a messy codebase to be quite rewarding in its own way.
It really helped to understand why SOLID principles are used.
Legacy projects usually are legacy because they have a river of money flowing through them. Making changes there is high-stakes work!
and that's why I'm a senior engineer since my first internship!
I always considered legacy projects more interesting than zero to one. A lot less reinventing the wheel and bikeshedding, a lot of puzzles - painful ones, sure, but at least it's not a bunch of CRUD (or the usual startup experience, which is a constant flux of rewriting-crud-very-fast)
Any project that has been successful for long enough will eventually become a legacy project. So I guess it's not wrong.
Bonus points if you got to watch your own greenfield work turn into a legacy project someone else maintains.
Sometimes junior engineers (of all ages) manage to green-field new project that is legacy in about a week.
Does replacing a completely unsalvageable legacy project count? It's done and dusted at this point, but I replaced a C/PHP codebase written by an IP-trolling freelancer who kept project sources as his personal IP, won back after a legal settlement. It was all deliberately obfuscated single letter naming and misdirected logic paths, along with a heavy reliance on global state. Investigating any individual part was manageable, thankfully, but I still had to unwind a great many useless stack frames and reverse-engineer several intense procedural bit-logic algorithms.
But but... ant was used even in the first few Android SDKs! It's practically new and shiny!
I have the feeling that I’ve mostly worked in legacy projects. What does it say about me?
You've skipped over "senior" and have become an honoured "legacy" engineer.
Good point.
Legacy is when you have a shell script call an ant script that calls shell scripts that call make scripts that fire c precompile requests to a remote database. (Of course it's now packaged in docker because it runs on a version of linux that makes the infosec guys come visit you at your desk)
Oh and every subsystem shares dependency references that are only used by one subsystem.
Oh, and the C includes are wrong and overwritten by the shell scripts and make files.
GOOD LUCK
OH, and your team is too scared to use the simplified build because there are too many things they don't understand, even though you did hash comparisons for every built artifact.
does everyone hate it? maybe I found my niche then because it’s actually pretty fun because you get to hack it up and come up with creative solutions, it’s like Indiana Jones exploring for lost artifacts, with all the perils
Could be fun with a decently made legacy system without artificial pressure from POs, managers, etc., but hell when you inherit a mess and nobody (who matters) believes you when you say you need to first fix 12 issues before you can do something simple.
What if your entire career is working on legacy projects? ;-;
What about if you worked on legacy work but using AI?
Some would say legacy code is code without unit or functional tests. I wouldn’t classify that as senior engineer worthy. Maybe masochism
(2023)
what about if ALL you did was work on legacy project? :)
Oh boo hoo, I had to deal with Ant.
Legacy is a good one.
Others:
Big code base. Big team. Big company. Different languages (and preferably in different paradigms; OO/FP and low/high-level). Different frameworks/stacks. And "in anger" also really helps IMHO (this means when shit went really bad and needs to be fixed fast).
>Using singletons wasn’t exactly a bad idea in this case since most of the code wasn’t covered with unit tests.
Huh?
[dead]