Looking for books where an alien/ancient/unexplainable structure is explored

I love these kinds of books. A few suggestions:

• Randezvouz With Rama by Arthur C. Clarke
• Roadside Picnic by Arkady and Boris Strugatsky
• Solaris by Stanisław Lem
• House of Leaves by Mark Z. Danielewski (not sure about this one, haven't read it yet.)
• Ringworld by Larry Niven

Hope you find something that fits your tastes and looking forward to other suggestions.

Sphere, by Michael Crichton

The Hyperion series, particularly the first book has some of that. And can be read without reading the other books.

A Deadly Education series is pretty much this to an extent. And a series I love!

The Silo series is adjacent to this, I love all three books.

The Expanse series has a few books that fit the bill but you really can't skip to those books so it may not be what you're looking for.

Annihilation has some freaky/disturbing unknown structure exploration in it.

Iain M. Banks created a whole universal alien system called "The Culture" which are mostly sentient machines. Also woven in is how humans evolved. There are several books of his that exist in the universe of "The Culture". I haven't read his stuff in a while but remember being intrigued while reading them.

Jack McDevitt writes a series which is basically space archaeology. You may like those. I think Engines of the Gods is thr first one.

"Piranesi" or "There Is No Antimemetics Division"

Though the later is more about exploration of hard to grasp cognitive concepts than a structure.

There's a few authors who carried on the Lovecraftian tradition - August Derleth, Clark Aston Smith, Robert Bloch (although his most notable work is not Lovcraftian - he wrote the book Psycho, later filmed by Hitchcock). Later on Ramsey Campbell and Brian Lumley.

Then there's the kind of post-Lovcraftian stuff - Thomas Ligotti is the big name in that area.

There's also a few modern authors who tried to bring modern sensibilities to Lovecraft's work. Ruthanna Emrys' re-imagining of the Innsmouth based events for example.

In Blindsight, a spaceship crew explores a huge and incomprehensible alien spaceship. While the book has some interesting thoughts on sentience and intelligence, the plot makes no sense and the crew are cruel sociopaths.

Adrian Tchaikovsky

Walking To Aldebaran.

Astronaut finds an artifact orbiting past Saturn...

Maybe not exactly what you're looking for, but There is no Antimemetics Division by QNTM revolves around ideas / beings / structures / civilizations that are literally unknowable.

That's a pretty common feature in Alastair Reynolds's works, though it's not always the primary focus.

The "Revelation Space" series of books features that trope pretty heavily, and it's also plot relevant. There's also the "Revenger" series where the main characters' job is to explore ruins from failed "turnover civilizations" and extract valuable loot to sell and fund their expeditions.

If manga counts, Blame has an absurdly enormous anomalous structure.

I'm going to give you a list that may be more on the fringes of what you want.

You might be interested in Clive Barker's The Great and Secret Show, and its sequel Everville. It's been a while since I've read them. I think Everville had more of the eldritch stuff, but I don't recall whether it holds up on its own.

Peter Cline's 14 fits what you want, though it has a bit more of a modern tone. The Fold takes place in the same world.

American Elsewhere by Robert Jackson Bennet.

The Way Up Is Death by Dan Hanks

An Absolutely Remarkable Thing by Hank Green

14 by Peter Clines is phenomenal and right up your alley. It is the first of a series and I think the second one is best and 3rd one is completely optional to read as far as the over arching plot.

Since you read German: I highly recommend Die Stadt der träumenden Bücher from Walter Moers.

It takes a bit to get there but after the first part of reaching Buchhaim, then the catacombs below it are the main focus. And for me the exploration of the unknown areas there makes it my favourite book.

Charlie and the Chocolate Factory.

Ursula le Guin has a bunch in the Hainish Cycle I can think of. My favourite of those is Vaster than Empires and More Slow.

Rimworld by Larry Niven?

Rogue Moon by Algis Budrys. A lone astronaut explores a maze on the moon that kills people who break arbitrary and unexplained rules.

Just FYI, the genre you're describing is called BDO (Big Dumb Object).

The Stone Builders is good. It's about a team of scientists exploring the newly discovered remains of an underground city on a colony planet with no known native sentient life.

Saw House of Leaves and Annhilation mentioned. Didn't A Wrinkle In Time have a bit of this as well?

Since you mention Reynolds you may also be interested in Diamond Dogs. Probably quite different from Eversion though and a short story instead.