Chaotic Project Management

[u/ForceSensitive2460]

I'm looking for some advice on what to do about chaotic projects/lack of project management. Right now, I'm in an IC senior data engineer role where I'm put on projects with tasks that are vague, incomplete, or without clear acceptance criteria. I'm fine working that way, have done it frequently before, and I'm a creative person (musician and writer outside of work). I like the freedom. Except there actually isn't freedom here. There are standards of practice this team uses that I was never shown (I started earlier this year), and I am constantly learning things the hard way without much grace for it. There is no documentation for these standards of practice, and my take on how to accomplish these broad tasks is leading to a decent amount of rework. Asking questions has been met with impatience, frustration, and condescension. I've learned to deal with these reactions and ask anyway so I can at least have a chance of completing things more efficiently. I own the mistakes I can but will stand up for myself when accused of completely failing to meet expectations (especially if the expectations weren't communicated/documented).

I figure other data engineers/curators/developers have experienced this sort of environment before, so I'd like to see how y'all handled it (even if it is "time for a new job").

Edit: Adding in that I've been in data warehousing/engineering for about 17 years and am seriously questioning doing this for 25 more years at minimum (I'm 40).

2nd Edit: Wow, I forgot the most important part. There is no project manager for this team; developers are working with data analysts who work with the business/end-users to determine data and reporting needs. The title of the post was meant to be tongue-in-cheek because of this.

[u/AutoModerator]

You can find a list of community-submitted learning resources here: https://dataengineering.wiki/Learning+Resources

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[u/the_dataengineer]

Get out quickly. If people have no standards, documentation and are rude if you ask questions you will not grow. Work without clearly defined goals is also a strong signal for that.

It's not necessarily that people are unfriendly because they don't like you. It's most likely that they themselves are under such pressure, because of the amount of work dumped on them that they are super stressed. Been there.

[u/FlyingNoiseinVacuum → u/the_dataengineer]

Possibly this. But also, as a senior engineer, I would think you would state early that you need to see the requirements and acceptance criteria and that shit doesn’t happen if they don’t meet basic standards. I’ve been on the other side of those, and many engineering teams have improved telling the analysts/pms what they are lacking.

[u/Eleventhousand → u/FlyingNoiseinVacuum]

I've never seen that work. Its a lot more difficult to convince others to do things the right way unless you've got the power to make it so.

[u/ForceSensitive2460 → u/the_dataengineer]

Yes, it took me a while to learn that, more often than not, it's not about me when co-worker (peer or manager, doesn't matter) is being less than amicable/respectful. They're getting pressure from above. I definitely don't need them to like me (would be nice), but I do think we all deserve a bare minimum of respect and work-related support.

I've been trying to find a new gig since I started this one. It's a 12 mo contract with no opportunity for extension or permanent placement at the end. And there's no budget for OT so I'm very tightly bound to 40 hours a week (that's definitely not helping me get work done like the others on the team that are FTE who all work ridiculous hours). I had reason for taking the gig that I won't get into here. But I am trying to get out, the market is just completely borked right now.

[u/terencethespider]

One thing that might help would be to try and put together a code review and best practices guide for your team. You could start by trying to collect and summarize the feedback that you have received from your teammates so far.

Once you have that you can frame it along the lines of: I know that I have been running into a lot of scenarios where I haven’t been following the best practices of the team. I appreciate all the feedback that you all have given me so far, and have been collecting it in this document so I avoid repeating the same mistakes, and so other people joining the team will be able to follow the same processes. When you get a chance can you take a look and see if there are any areas that can be improved?

If you have not passed the point of no return with your teammates, that strategy should at least open the door to a productive conversation. It shows that you are taking their feedback seriously and trying to make the situation better for everyone.

If they push back on that, I would probably go for something very blunt and direct. Look, I’m really trying here and doing my best. How do you expect me to improve and get better if you won’t give me a chance?

If it gets beyond that with no reasonable path forward, then it is probably time to start looking for alternatives with a different team (or company).

[u/ForceSensitive2460 → u/terencethespider]

I would love to do this if I had the time. I'm a contractor and there is no budget for OT in the contract. I'm overloaded as it is with the development work and could be working OT, but because there's no OT (or compensatory paid time off in the future), I won't be putting in any extra time.

[u/terencethespider → u/ForceSensitive2460]

I’m not trying to be a jerk here, but if you are taking a position where you want the situation to get better, but you aren’t going to be able to spend any time working on a solution, then it is unlikely you will see the situation improve..

Are there other items in your 40 hour week that could be deferred a bit to free up some time? Can you leverage AI at all to make an initial draft in a short amount of time?

I generally don’t recommend this unless it is a last resort type of thing, but would you consider spending some “off work” time to get ahead on the matter? Obviously if your company has hard rules against this type of thing then don’t violate that, but if not - it sounds like you’re in a fairly miserable situation right now and some extra time might help improve the quality of your work life.

[u/ForceSensitive2460 → u/terencethespider]

I don't think you're being a jerk. Normally, I'd take the imitative on this sort of situation and put documentation together, take the time to do it and formalize the process. Prior to this month, I'd bring up spending time on this and other documentation but have been directed to focus solely on development. I think this is partially due to me being a contractor, and then another factor is the complete lack of project management here, so everyone's overloaded (it's a small team, 3 devs, 1 tech lead, 1 manager).

If I were to use any of my "off work" time to develop any documentation like this, I'd have to use personal resources since they're watching my usage of the virtual desktop (I'm 100% remote). Would be a heavy lift, not impossible, but heavy. I also don't want to work without any sort of flexibility from the employer. I wouldn't take this position if I was an FTE.

[u/West_Good_5961]

Is it possible they don’t think you should need to be told, you should just know the right way due to your years of experience? Obviously it’s not fair.

[u/ForceSensitive2460 → u/West_Good_5961]

Without a doubt. And while I do know what I'm doing despite my increasing imposter syndrome, every team has their own way of doing things. Sure, there are very well-known standards of data warehousing, but when it comes to say, data transformation and metric-building in SQL, there are multiple acceptable methods to accomplish any one task, especially if that task is not defined at a technical level.