Table of Contents

File Transfer - Wget

About

wget is a headless browser command line.

It's most used to download files from the Web but you can also download/mirror a whole website.

It supports the HTTP, HTTPS, and FTP protocols

Windows

Windows 10 PowerShell has also curl available.

Installation

Git_Home\usr\bin

Management

Download to a specific file name

wget <file.ext> -O /path/to/folder/file.ext

Download to a specific folder

wget <file.ext> -P /path/to/folder

Download a file with basic authentication

What is and how works the Basic Authentication scheme? (HTTP) 

wget --http-user=login --http-password=password https://example.com/file.pdf

Download a whole web site

wget 
     --recursive \ # Download the whole site.
     --page-requisites \ # Get all assets/elements (CSS/JS/images).
     --adjust-extension \ # Save files with .html on the end.
     --span-hosts \ # Include necessary assets from offsite as well.
     --convert-links \ # Update links to still work in the static version.
     --restrict-file-names=windows \ # Modify filenames to work in Windows as well.
     --domains yoursite.com \ # Do not follow links outside this domain.
     --no-parent \ # Don't follow links outside the directory you pass in.
     https://yoursite.com/whatever/path # The start URL to download

Options: 

-r - recursive
-m, --mirror            Turns on recursion and time-stamping, sets infinite recursion depth, and keeps FTP directory listings.
-p, --page-requisites   Get all images, etc. needed to display HTML page.
-E, --adjust-extension  Save HTML/CSS files with .html/.css extensions.
-k, --convert-links     Make links in downloaded HTML point to local files.

It can be used to pre-render

Documentation / Reference