# gallery-dl-archive-manager

Scripts to manage a (currently twitter only) archive using gallery-dl. Much of the code came from a need to augment pre-existing, outdated archives that were originally created from the `twittermediadownloader` browser extension.

## Config
This repo uses its own `config.json` in order to save media in the same format as `twittermediadownloader`. The scripts depend on the media being saved in this format.

## Scripts

### `node run-initDb.js --path={/path/to/your/archive} --args={gallery-dl args}`

Initializes a user database from existing folders. Useful if you have a pre-existing archive of users.

### `node run-downloadDb.js --path={/path/to/your/archive} --threads={#} --args={gallery-dl args}`

## Args

Standard args:

### `--path={/path/to/your/archive}`

The path to the archive. This is a parent directory with a list of child directories which correspond to users.

### `--threads={#}`

Max number of concurrent download threads. Only this number of concurrent gallery-dl download threads will run at a given time, other remaining users will be queued.

### `--args={gallery-dl args}`

Additional args to pass to gallery-dl. Note that these aren't currently checked for duplicates that may be used by this repo.

## TODO

### `run-addUsers.js --path={/path/to/your/archive} --users={string array of user(s)} --threads={#}`

Should add a new user to the db and initiate a full download similar to `run-downloadDb.js`

### `run-updateDb.js --path={/path/to/your/archive} --mode={search|media} --threads={#} --args={gallery-dl args}`

Should pull from the user database and update the archive without doing a full download.

- `--mode`
  - `=search`: The DB should save with a lastUpdated field. This should be used as a date for the /search API. Preferred if it's been a long time since an update has happened for a user and/or the user has uploaded a significant amount of media since lastUpdated.
  - `=media`: This will run /media and stop after hitting skipped files, as skipped files indicate hitting lastUpdated. Preferred in normal circumstances.