[Linux/browser] Can I extract a session token/cookie to use with wget or curl?

A cloud provider's web GUI eats all my RAM when downloading big files. I would like to download the files using wget or curl instead, but I assume I'm going to need some cookie or token in the argument URL that authenticates the request?

Is there any browser add-on or other means to extract said information so that I can copy paste it into the terminal?

15 points · 6 comments · view on lemmy.world

6 Comments

hades@feddit.uk · 17 pts · 1d

You can open developer tools, download one of the files, then right click the request in the Network tab and select “Copy as curl command”, this will generate a curl command with all the headers and cookies.

MummifiedClient5000@feddit.dk · 5 pts · 1d

If the token is sent in the header of the request (which is common) then Firefox and Chrome can export the exact request as a curl command from the dev console network tab.

Successful_Try543@feddit.org · 4 pts · 1d

Yes, but you need to install a browser addon first, that allows exporting the cookies as txt files.

https://scrapingant.com/blog/wget-cookies

BehemothExplorer@sh.itjust.works · 4 pts · 1d

I know almost nothing about this sort of thing, but I have managed to do what you're describing without an add-on. Go to where you go in the cloud to download. Open your browser console (Ctrl+Shift+C or F12) and go to the Networking tab. Filenames will appear in the Networking tab as you interact with the cloud through the browser. If you right-click on a Filename, a dialogue box for it will open. The first option is Copy Value, which will open a sub-menu that includes Copy as cURL. This option copies everything that your browser sent to the cloud (to get the file you right-clicked) to your clipboard as a cURL command, which you can then paste into a bash script.

This won't work for every cloud provider, but it does for some. Add-ons can be convenient, but there's something to be said for "popping the hood," so to speak. Good luck!

epyon22@sh.itjust.works · 1 pts · 1d (1 reply)

Depending on how they are downloading to your computer you may not have a single URL to download a file, which will increase the effort to do this from curl/wget. Given you are complaining about RAM usage they might have a custom multi-part downloader and assemble the file within the browser's logic. I've seen this on sites like MegaUpload before.

akunohana@piefed.blahaj.zone · 1 pts · 1d

Yeah, I feel like my bigger files are now trapped on their servers :(