Commit Graph

292 Commits

Author SHA1 Message Date
torzdf
696692dc08 Fixups
- Deprecations - display correct long argument
  - Minor spelling + linting
2024-04-21 16:22:42 +01:00
torzdf
d75898f718 Bugfixes for video file alignments storage:
- extract/convert: Load images with correct video extension
  - Manual tool: Cache thumbnails with correct extension
  - Mask tool + Preview tool:: Update legacy alignment keys for pre-video extension storage
2024-04-18 12:45:33 +01:00
torzdf
957734dfc0 convert: Bugfix: Update legacy .png video alignments to include video file extension 2024-04-17 12:37:47 +01:00
torzdf
1c081aea7d
Add ability to export and import alignment data (#1383)
* tools.alignments - add export job

* plugins.extract: Update __repr__ for ExtractorBatch dataclass

* plugins.extract: Initial implementation of external import plugins

* plugins.extract: Disable lm masks on ROI alignment data import

* lib.align: Add `landmark_type` property to AlignedFace and return dummy data for ROI Landmarks pose estimate

* plugins.extract: Add centering config item for align import and fix filename mapping for images

* plugins.extract: Log warning on downstream plugins on limited alignment data

* tools: Fix plugins for 4 point ROI landmarks (alignments, sort, mask)

* tools.manual: Fix for 2D-4 ROI landmarks

* training: Fix for 4 point ROI landmarks

* lib.convert: Average color plugin. Avoid divide by zero errors

* extract - external:
  - Default detector to 'external' when importing alignments
  - Handle different frame origin co-ordinates

* alignments: Store video extension in alignments file

* plugins.extract.external: Handle video file keys

* plugins.extract.external: Output warning if missing data

* locales + docs

* plugins.extract.align.external: Roll the corner points to top-left for different origins

* Clean up

* linting fix
2024-04-15 12:19:15 +01:00
torzdf
95b4431c57 Deprecate multi-character cli switches 2024-04-05 13:51:57 +01:00
torzdf
70c064ca7d Minor fixups and linting 2024-04-03 15:14:32 +01:00
torzdf
983901466f linting: Reduce Class min-public-methods and fix spacing 2024-04-03 14:37:48 +01:00
torzdf
a9d87ae007 linting: Remove pylint hint for logger 2024-04-03 14:03:54 +01:00
torzdf
e80ae046d5
Convert - Patch writer (#1352)
* Add cli-args and config opts for patch writer
* Add supporting code to convert funcs
* writers
  - move frame count functions to _base
  - Add kwargs for pre_encode
* scripts/lib convert - add code for patch writer
* lib.image.encode_image - Add cv2.imencode args
* patch_defaults: Add face index location option
* Add patch writer with PNG support
* Send correct matrix to patch plugin
* Add canvas origin option
* Add Tiff format to Patch Writer
* Docs and locales
* Add ROI to output
* convert: choose warp border by face count
2023-09-28 11:55:38 +01:00
torzdf
cf0efff6ba
Learning Rate finder (#1341)
* Add LR Finder support structure

* Move trainer.Feeder to lib.training.generator

* Expose model.io

* Add lr_finder

* Update docs and locales

* Pre-PR fixups
  - Fix training graph not displaying
  - CI fixes
  - Switch lr finder progress to tqdm
  - Exit lr finder early on NaN
  - Display lr finder progress in GUI
2023-08-22 17:07:36 +01:00
torzdf
6a3b674bef
Rebase code (#1326)
* Remove tensorflow_probability requirement

* setup.py - fix progress bars

* requirements.txt: Remove pre python 3.9 packages

* update apple requirements.txt

* update INSTALL.md

* Remove python<3.9 code

* setup.py - fix Windows Installer

* typing: python3.9 compliant

* Update pytest and readthedocs python versions

* typing fixes

* Python Version updates
  - Reduce max version to 3.10
  - Default to 3.10 in installers
  - Remove incompatible 3.11 tests

* Update dependencies

* Downgrade imageio dep for Windows

* typing: merge optional unions and fixes

* Updates
  - min python version 3.10
  - typing to python 3.10 spec
  - remove pre-tf2.10 code
  - Add conda tests

* train: re-enable optimizer saving

* Update dockerfiles

* Update setup.py
  - Apple Conda deps to setup.py
  - Better Cuda + dependency handling

* bugfix: Patch logging to prevent Autograph errors

* Update dockerfiles

* Setup.py - Setup.py - stdout to utf-8

* Add more OSes to github Actions

* suppress mac-os end to end test
2023-06-27 11:27:47 +01:00
torzdf
03f5c671bc
Remove plaidML support (#1325)
* Remove PlaidML reference from readme files

* Remove AMD option from installers

* remove amd requirements and update setup.py

* remove plaidml test from CI workflow

* gpustats: remove plaidml backend

* plaid removals:
  - faceswap.py - python version check
  - setup.cfg - plaidml typing ignore
  - lib.keras_utils - All plaid code
  - lib.launcher.py - All plaidml checks and configuration

* remove tf2.2 specific code from GUI event reader

* lib.model - remove all plaidml implementations

* plugins.extract - remove plaidml code

* plugins.train remove plaidml code

* lib.convert - remove plaidml code

* tools.model: remove plaidml code

* Remove plaidML tests from unit tests

* remove plaidml_utils and docsting cleanups

* Remove plaidML refs from configs

* fix keras imports
2023-06-21 12:57:33 +01:00
torzdf
82e927d3e7 bugfix: Preview tool Don't error if too few images 2023-06-18 11:38:43 +01:00
torzdf
4a5a10977b Bugfix: Disable batch mode patch for non-Linux machines 2023-02-10 18:02:14 +00:00
torzdf
8a5ee6d9bf Bugfix: Patch extract memory leak in batch mode 2023-02-10 15:52:34 +00:00
torzdf
a1ef5edd39 tests:
- unit test: tools.alignments.media
  - Add mypy test
  - Typing fixes
2023-01-21 00:41:18 +00:00
torzdf
dc94ed9cad
DirectML Support (#1291)
* DML Support
  - Add requirements. Update setup.py. Update Windows Installer
  - Add 'directml' backend. Update backend supporting logic
  - GPUStats - Add GPU querying from Win SDK for DML backend
  - Windows PlaidML - Deprecation Warning
  - Remove deprecated '-d' training flag
  - Logging in DML functions
- Unittest
  - Windows Unit Test
  - lib.utils
2023-01-06 14:45:57 +00:00
torzdf
9e2026f6fe Extract: Implement re-align/2nd pass
- implement configurable re-align function in extract
  - update locales + documentation
  - re-factor align._base and split to separate modules
  - move normalization method to plugin parent
  - bugfix: FAN use zeros for pre-processing crop
  - lint AlignedFilter
2022-11-22 00:37:29 +00:00
torzdf
e3b457693e bugfix - extract filters: spaces in folder names 2022-11-10 01:41:40 +00:00
torzdf
0f5d2e887c Face-filter updates:
- Allow selecting folder as well as multiple images
  - Lower default threshold and update helptext
  - bugfix: detector error when using all aligned faces
  - Standardize output folder name
2022-11-04 18:43:55 +00:00
torzdf
1d1face00d Update Face Filter
- Remove old face filter
  - plugins.extract.pipeline: Expose plugins directly
  - Change `is_aligned` from plugin level to ExtractMedia level
  - Allow extract pipeline to take faceswap aligned images
  - Add ability for recognition plugins to accept aligned faces as input
  - Add face filter to recognition plugin
  - Move extractor pipeline IO ops to own class
2022-10-31 18:25:32 +00:00
torzdf
2e8ef5e3c8 GUI - Preview updates
- Training preview. Embed preview pop-out window
  - Bugfix - convert/extract previews
2022-10-20 18:51:39 +01:00
torzdf
dab823a3eb Typing - lib.gui.display_command 2022-10-17 18:14:04 +01:00
torzdf
f32f460e3f Refactoring
- Change saved previews to PNG
  - Split lib.gui.utils
  - Disable flake8 on __init__ imports
2022-10-15 19:17:24 +01:00
torzdf
f1e3339fbe Add vggface2 to extraction pipeline 2022-10-10 13:09:02 +01:00
torzdf
220335f55d bugfix: Extract: Add video_meta to alignments dict 2022-09-26 22:51:25 +01:00
torzdf
1c8eb24b38 Bugfix: Correct naming of alignments file for videos with dots in filename 2022-09-23 14:14:29 +01:00
torzdf
de11b4c189 Bugfix: Extract. numbering issue when binning results 2022-09-23 01:37:51 +01:00
torzdf
50d23792ca Aligner filter - set sensible defaults 2022-09-21 01:04:33 +01:00
torzdf
71726c1ff5 Extract: Add metric information to debug images 2022-09-19 02:14:09 +01:00
torzdf
f5fa5b4337 Extract filter: Allow saving of filtered images 2022-09-19 00:13:41 +01:00
torzdf
a8f22cc019 Extract updates:
- Default CPU detector to MTCNN
  - add basic Aligner false positive filters
  - Typing: align + plugins
  - Use specific AlignerBatch class for alignment
  -
2022-09-18 19:44:41 +01:00
torzdf
952d79922b Bugfixes:
- Extract - batch mode. Exclude folders with no images
  - Train. Trigger the correct preview/mask update from gui trigger
2022-09-13 18:54:01 +01:00
torzdf
7da2cc3dd2 Training - Use custom preview pop-out 2022-09-07 11:49:52 +01:00
torzdf
ae7793e876 bugfix: convert - Process final items on truncated batch 2022-09-02 12:42:52 +01:00
torzdf
13cfb3f39e extract: Add batch processing mode 2022-08-31 19:48:47 +01:00
torzdf
04b197ec5f convert: reduce RAM consumption 2022-08-27 08:13:45 +01:00
torzdf
1022651eb8 Bugfix: convert - Gif Writer
- Fix non-launch error on Gif Writer
  - convert plugins - linting
  - convert/fs_media/preview/queue_manager - typing
  - Change convert items from dict to Dataclass
2022-08-26 23:56:03 +01:00
torzdf
9e503bdaa2 bugfix: debug landmarks 2022-08-23 19:19:16 +01:00
torzdf
2beceffad9
Data Augmentation update (#1263)
- lib.detected_face
    - Subclass Masks for Landmark based masks
    - Add training mask propery + methods to DetectedFace
  - lib.training_training
    - subclass TrainingDataGenerator for training and preview data
    - Split cache into own module
    - Reduce thread count to 1 to prevent image corruption + data re-use
    - Process on largest model input/output size rather than stored image size
    - Size and crop masks during caching stage
    - Implement ring buffer for data flow
    - Fix preview reload bug
  - augmentation
    - typing
    - switch color aug order
    - better initialization
    - Fix warp + landmark warp to correctly apply at different image scales
    - Slightly improved warp caching
    - Don't store whether image is_preview. Handle all data as training images implicitly
  - plugins.trainer: Typing and fixes to work with trainingdata refactor
2022-08-21 18:59:45 +01:00
torzdf
26e26c6288 Update TF to 2.9
- Update TF to 2.7 to 2.9
    - Bump dependencies
    - Remove decode from pynvml calls
    - force keras predict functions to non-verbose
    - update tests
    - update Tensorboard logging
    - Update docs
2022-08-02 17:12:26 +01:00
torzdf
2ea05623bd Update Distibution Strategies:
- Add Central Storage Stategy
  - Deprecate 'distributed' cli argument
2022-07-19 18:23:16 +01:00
torzdf
c8122bc499 bugfix: Stop preview window from stealing focus 2022-07-06 11:15:51 +01:00
torzdf
3c73ae4ec9 bugfix: Update preview screen in GUI 2022-07-01 19:06:42 +01:00
torzdf
7b9fc0454d Live Preview - Replace cv2 with matplotlib viewer 2022-07-01 13:07:01 +01:00
torzdf
b1a8183ab4 scripts.train - type checking 2022-06-30 18:41:58 +01:00
torzdf
c279ac3518 Disable saving optimizer state option 2022-06-24 13:43:26 +01:00
torzdf
06468c97d4 Training: Add setting option to save optimizer weights 2022-06-23 14:54:09 +01:00
torzdf
0d23714875 bugfix: extract - stop progress bar from going over max value 2022-05-16 23:26:42 +01:00
torzdf
b11ed6ead0 typofix 2022-05-13 12:10:51 +01:00