I do like to have a look at these type of overviews, but I highly doubt they're actually useful to anyone. Stuff on there is so random it'll probably confuses the people this is intended for more than it helps..
Hmm good question.. I do like overviews with tooling/technology; what is new, what is left behind, what is considered standard at the moment, etc. It's just that, since DE is such a broad field, I think it would be better to let the field you're active in determine what to focus on.
If the goal is to provide junior DE's with some guidelines, I prefer to educate them on what would functionally work best to solve a problem, so they can make the right decision on what tool to pick (where in practice, this is likely already decided; you just work with what's there).
7 Comments
nydas@lemmy.ml · 1 pts · 3y
Isn’t Apache Arrow a table format rather than a batch processing tool?
You could add SQLMesh as an alternative to dbt.
ephemeral404@lemmy.ml · 1 pts · 3y
thanks for suggesting sqlmesh. regarding arrow, have you read this - https://arrow.apache.org/docs/format/Columnar.html
daanzel@lemmy.world · 1 pts · 3y
I do like to have a look at these type of overviews, but I highly doubt they're actually useful to anyone. Stuff on there is so random it'll probably confuses the people this is intended for more than it helps..
ephemeral404@lemmy.ml · 1 pts · 3y
what could be a better approach?
daanzel@lemmy.world · 1 pts · 3y
Hmm good question.. I do like overviews with tooling/technology; what is new, what is left behind, what is considered standard at the moment, etc. It's just that, since DE is such a broad field, I think it would be better to let the field you're active in determine what to focus on.
If the goal is to provide junior DE's with some guidelines, I prefer to educate them on what would functionally work best to solve a problem, so they can make the right decision on what tool to pick (where in practice, this is likely already decided; you just work with what's there).
Deebster@lemmy.ml · 1 pts · 3y
btw, you've spelt Engineering wrong in the sidebar.
ephemeral404@lemmy.ml · 1 pts · 3y
thanks for pointing out. fixed.