Commit Graph

810 Commits

Author SHA1 Message Date
torzdf
f3b88d5626 bugfix: Get correct output size for learn mask 2022-08-29 01:04:40 +01:00
torzdf
05077265d7 utils.debug_time - thread support 2022-08-28 16:41:52 +01:00
torzdf
2a7c18ac1d bugfixes:
- lib.training: Correct  input + output size for pre-existing models
  - lib.align.detected_faces - fix trace logging
  - lib.util.debug_times
2022-08-28 12:25:30 +01:00
torzdf
1022651eb8 Bugfix: convert - Gif Writer
- Fix non-launch error on Gif Writer
  - convert plugins - linting
  - convert/fs_media/preview/queue_manager - typing
  - Change convert items from dict to Dataclass
2022-08-26 23:56:03 +01:00
torzdf
326110f09d bugfix - timelapse image loader
multithreading.py - typing + docs
2022-08-25 12:38:05 +01:00
torzdf
9e503bdaa2 bugfix: debug landmarks 2022-08-23 19:19:16 +01:00
torzdf
66845ea5f0 bugfix: Alignments - Thumbnail generation check 2022-08-22 10:41:33 +01:00
torzdf
2beceffad9
Data Augmentation update (#1263)
- lib.detected_face
    - Subclass Masks for Landmark based masks
    - Add training mask propery + methods to DetectedFace
  - lib.training_training
    - subclass TrainingDataGenerator for training and preview data
    - Split cache into own module
    - Reduce thread count to 1 to prevent image corruption + data re-use
    - Process on largest model input/output size rather than stored image size
    - Size and crop masks during caching stage
    - Implement ring buffer for data flow
    - Fix preview reload bug
  - augmentation
    - typing
    - switch color aug order
    - better initialization
    - Fix warp + landmark warp to correctly apply at different image scales
    - Slightly improved warp caching
    - Don't store whether image is_preview. Handle all data as training images implicitly
  - plugins.trainer: Typing and fixes to work with trainingdata refactor
2022-08-21 18:59:45 +01:00
torzdf
5e73437be4 lib.align updates:
- alignments.py
    - Add typed dicts for imported alignments
    - Explicitly check for presence of thumb value in alignments dict
    - linting
  - detected_face.py
    - Typing
    - Linting
    - Legacy support for pre-aligned face
    - Update dependencies to new property names
2022-08-19 12:36:08 +01:00
torzdf
a2de4a9798 lib.align.aligned_face updates
- Typing
  - Legacy support for pre-aligned faces
  - Coverage support for pre-aligned faces
  - Standardized retrieval of sub-crops
2022-08-19 11:53:46 +01:00
torzdf
3295089737 lib.detected_face.Mask
- Add source + target offset and coverage to set_sub_crop method
2022-08-19 10:57:40 +01:00
torzdf
73111dda31 Minor updates:
- Update .pylintrc for opencv
  - Update setup.cfg for scipy
  - Typing update for lib.utils.DebugTimes
2022-08-19 10:13:50 +01:00
torzdf
c0d0f04b70 bufix: mask plugin: Fix error when no mask is selected 2022-08-18 19:35:50 +01:00
torzdf
26dde3c19a Add CPU option to BiSeNet and MTCNN
- Add CPU option to KSession
  - MTCNN optimizations
  - Update docs for bisenet + mtcnn
2022-08-08 14:29:32 +01:00
torzdf
629c02a61e Add "custom" mask 2022-08-05 14:09:05 +01:00
torzdf
26e26c6288 Update TF to 2.9
- Update TF to 2.7 to 2.9
    - Bump dependencies
    - Remove decode from pynvml calls
    - force keras predict functions to non-verbose
    - update tests
    - update Tensorboard logging
    - Update docs
2022-08-02 17:12:26 +01:00
torzdf
92912a7061 Update setup.py
- Realtime output for Windows
  - color logging for compatible Windows versions
2022-07-30 13:45:09 +01:00
torzdf
03f6cb4e7e setup.py: implement logging 2022-07-28 23:53:31 +01:00
torzdf
54398559ff Set default distribution strategy to 'default' 2022-07-20 12:29:14 +01:00
torzdf
2ea05623bd Update Distibution Strategies:
- Add Central Storage Stategy
  - Deprecate 'distributed' cli argument
2022-07-19 18:23:16 +01:00
torzdf
e362fd58c5 Add DNY presets to Phaze-A 2022-07-17 18:05:06 +01:00
torzdf
7e0dbcdaa7 Update docs 2022-07-17 17:54:22 +01:00
torzdf
582c2ce40c Add Flip Loss Function
- Add Flip for AMD and TF
    - Split Perceptual Loss functions to own modules
    - Fix allowed input shape for models
    - Allow GUI tooltip to display at higher width
2022-07-07 01:02:11 +01:00
torzdf
7b9fc0454d Live Preview - Replace cv2 with matplotlib viewer 2022-07-01 13:07:01 +01:00
torzdf
91fecc47b2 lib.Utils - add DPI detector 2022-06-30 13:34:10 +01:00
torzdf
6e0360150d bugfix: Correctly show installed TF Version on AMD error 2022-06-29 18:10:47 +01:00
torzdf
eb3c612f29 Replace restore tool with model tool 2022-06-28 14:27:23 +01:00
torzdf
ef5451e4f1 bugfix: Convert revert warp border to transparent 2022-06-28 10:47:49 +01:00
torzdf
8ec1e1a091 Nvidia: Add AutoClip training option 2022-06-23 18:00:22 +01:00
torzdf
c3c3483eed AMD Bugfixes: Save Optimizer Weights 2022-06-23 15:55:24 +01:00
torzdf
06468c97d4 Training: Add setting option to save optimizer weights 2022-06-23 14:54:09 +01:00
torzdf
66c2b7b979 Bugfix: Stats elapsed time 2022-06-22 19:20:51 +01:00
torzdf
da942e3b3f Nan bugfix in Stats 2022-06-21 12:32:09 +01:00
torzdf
5d700e869e stats bugfixes 2022-06-20 13:07:37 +01:00
torzdf
ad408f07fb typofix 2022-06-20 12:46:23 +01:00
torzdf
9e94273f89 bigfix - Live graph reading 2022-06-20 12:04:35 +01:00
torzdf
e2fc070370 Bug fixes
- PhazeA tooltip spacing
  - Graph live cache bug
2022-06-20 03:38:47 +01:00
torzdf
d7ffcda3b0 Add LPIPS loss function 2022-06-19 18:42:34 +01:00
torzdf
bad5025aea Core updates
- Change loss loading mechanism
   - Autosize tooltips based on content size
   - Random linting + code modernisation
2022-06-19 12:32:39 +01:00
torzdf
f2e6f24651 Centralize model storage 2022-06-18 19:54:02 +01:00
torzdf
ef79a3d8cb Add AlexNet + SqueezeNet definitions 2022-06-18 18:21:41 +01:00
torzdf
1d434b73a4 Enable LogCosh Loss for AMD 2022-06-18 16:12:39 +01:00
torzdf
84b47fde95 linting 2022-06-18 15:36:39 +01:00
torzdf
42407e9d5c linting 2022-06-18 14:52:18 +01:00
torzdf
308c5edf39 Loss updates
- Split Loss to own package
  - Add Focal Frequency Loss (Nvidia only)
  - Enable GMSD for AMD
2022-06-18 14:33:45 +01:00
torzdf
9d55ade27f amd gpu-stats bugfix 2022-06-18 11:16:03 +01:00
torzdf
d9c84a5f9f Add Laplacian Pyramid Loss 2022-06-18 02:29:19 +01:00
torzdf
04337e0c5e SSIM Updates
- Standardize DSSIM Function
  - Implement MSSIM function for AMD
2022-06-18 00:02:28 +01:00
torzdf
94c3dcff7e Training updates
- Add multiple selected loss functions
  - Unlock loss as a model configuration
  - Phaze-A remove encoder scaling max xap
2022-06-17 20:24:26 +01:00
torzdf
98a65277d8 Fix AMD Tests + docs 2022-06-17 17:59:55 +01:00
torzdf
917acaa452 bugfixes
- Fix MS-SSIM on multi-gpu
  - Swallow print bug on multi-gpu
2022-06-12 02:04:11 +01:00
torzdf
3d8e674adc convert - Fix affine borders 2022-06-08 12:44:05 +01:00
torzdf
73442b520c convert: remove box blend plugin 2022-06-08 00:54:47 +01:00
torzdf
20a657d6cf Bump minimum TF Version 2022-06-07 10:54:51 +01:00
torzdf
5c9fa1aa03 Train updates
- Output full model summary last
  - Fix activations in Phaze-A upscales
  - Phaze-A Add option to place some upscales in fc model
2022-06-07 01:11:26 +01:00
torzdf
c7e577e887 Phaze-A - Updates for future implementations 2022-06-06 13:15:21 +01:00
torzdf
a99049711f Model updates
- Increase model summary width
- Phaze A updates
  - Update some min/max values
  - Add Decoder Filter Slope Mode
  - Add additional arguments for Upsampling2D
  - Adjust upsampling method for multiple upsamples in FC layers
  - Typing
2022-06-06 11:50:05 +01:00
torzdf
8acf3228ad nn_blocks - linting 2022-06-06 09:50:44 +01:00
torzdf
3a9764da9e lib.model.nnblocks - typing + cleanup upscales 2022-06-06 09:29:36 +01:00
torzdf
1301959907 Bugfix: 2 tf loss functions 2022-05-29 18:16:06 +01:00
torzdf
afec523093 Bugfixes:
- Stats graph - Handle NaNs in data
  - logger - de-elevate matplotlib font messages
2022-05-29 13:13:45 +01:00
torzdf
a2b8e324ae bugfix: distibuted training with dssim 2022-05-28 01:46:46 +01:00
torzdf
c2595c46d4 bugfix - add missing mask key to alignments on legacy update 2022-05-20 17:06:25 +00:00
torzdf
a5a598539c Manual tool - More robust handling of videos with duped frames 2022-05-18 00:30:56 +01:00
torzdf
b7e680c5a0 linting 2022-05-15 16:59:11 +01:00
torzdf
d6a3372eb0 Update matplotlib requirement
- bump min requirement to 3.5.1
  - Fix custom toolbar for v3.5.1
  - Documentation + linting
2022-05-15 16:48:58 +01:00
torzdf
988a171f04 Merge branch 'master' into m1 2022-05-12 15:57:38 +01:00
torzdf
5dfc9c03c9 bugfix: fix import order 2022-05-12 15:57:07 +01:00
torzdf
d83a39716b minor cleanup
- Consistent naming for backend selection
- Remove apple-silicon from setup.py (not yet implemented)
2022-05-12 12:40:53 +01:00
torzdf
f8c1bf26df Refactor apple-silicon
- Merge metal/__init__ into gpu_stats.apple_silicon.py
  - Use psutil.virtual_memory().available to get free memory
2022-05-12 12:12:00 +01:00
torzdf
d3f72c5449 Merge branch 'master' into m1 2022-05-12 12:02:34 +01:00
torzdf
fdb0a33c8d typofix - lib.gpu_stats.amd 2022-05-12 12:01:14 +01:00
torzdf
bcadc13c6d typofix - lib.gpu_stats._base 2022-05-12 11:49:40 +01:00
torzdf
a3e8fdc252
Merge pull request #1216 from geewiz94/patch-1
Add Apple Silicon (M1) support
2022-05-12 10:46:06 +01:00
torzdf
b057b719ce bugfix: prevent error on python 3.7 2022-05-12 08:53:13 +01:00
geewiz94
948f0fb82e Rename Apple Silicon backend to apple_silicon 2022-05-12 01:33:56 +02:00
geewiz94
6d6903d2ce Linting 2022-05-12 01:24:39 +02:00
geewiz94
a8b2b0d48e Merge branch 'merge-upstream-changes2' into patch-1 2022-05-12 01:21:37 +02:00
torzdf
bdbbad4d31
Refactor lib.gpu_stats (#1218)
* inital gpu_stats refactor

* Add dummy CPU Backend

* Update Sphinx documentation
2022-05-11 17:36:07 +01:00
geewiz94
24ab822e64
Merge branch 'deepfakes:master' into patch-1 2022-05-11 10:21:07 +02:00
torzdf
b2cd8eb867 bugfix: Windows - Stop training when terminate button pressed 2022-05-09 10:29:47 +01:00
torzdf
8ab085fae0 bugfix: gui - settings popup. Always reload config 2022-05-08 14:18:50 +01:00
torzdf
adb5975c94 Graph popup - Always open in same position 2022-05-08 10:22:27 +01:00
geewiz94
faef5a683e Cleanup 2022-05-07 13:50:10 +02:00
geewiz94
03a8b6228e Merge branch 'merge-upstream-changes' into patch-1 2022-05-07 13:12:42 +02:00
torzdf
a046248389 BugFix - lib.keypress 2022-05-06 17:15:18 +01:00
Dhyey Patel
feac8a0108
trainer: add support for non-interactive jobs (#1193)
Add support for training in non-interactive shell environment such
as Sun Grid Engine, Univa Grid Engine and others.

Tested on Univa Grid Engine.

Reference: https://stackoverflow.com/questions/967369/python-find-out-if-running-in-shell-or-not-e-g-sun-grid-engine-queue.
2022-05-04 23:07:31 +01:00
torzdf
332394edbf bugfix: correct init for ms-ssim amd 2022-05-04 10:03:14 +01:00
torzdf
5569abbb63 training - Add MS-SSIM loss function 2022-05-04 01:58:00 +01:00
torzdf
aa39234538
Update all Keras Imports to be conditional (#1214)
* Remove custom keras importer

* first round keras imports fix

* launcher.py: Remove KerasFinder references

* 2nd round keras imports update (lib and extract)

* 3rd round keras imports update (train)

* remove KerasFinder from tests

* 4th round keras imports update (tests)
2022-05-03 20:18:39 +01:00
torzdf
5adc5c536b ssim loss - Scale back 2022-05-02 18:19:00 +01:00
torzdf
c1512fd41d
Update code to support Tensorflow versions up to 2.8 (#1213)
* Update maximum tf version in setup + requirements

* - bump max version of tf version in launcher
- standardise tf version check

* update keras get_custom_objects  for tf>2.6

* bugfix: force black text in GUI file dialogs (linux)

* dssim loss - Move to stock tf.ssim function

* Update optimizer imports for compatibility

* fix logging for tf2.8

* Fix GUI graphing for TF2.8

* update tests

* bump requirements.txt versions

* Remove limit on nvidia-ml-py

* Graphing bugfixes
  - Prevent live graph from displaying if data not yet available

* bugfix: Live graph. Collect loss labels correctly

* fix: live graph - swallow inconsistent loss errors

* Bugfix: Prevent live graph from clearing during training

* Fix graphing for AMD
2022-05-02 14:30:43 +01:00
torzdf
cda49b3c3c Bugfix - Fix graphing not always showing loss for both sides 2022-04-16 15:13:46 +01:00
geewiz94
7dd1122c1c
Get free VRAM from Metal 2022-03-29 17:48:51 +02:00
Daniel Livingston
d6eedb09ff added apple silicon backend 2022-03-19 15:43:21 -04:00
Daniel Livingston
a7ef098083 keras import errors fix 2022-03-19 15:31:42 -04:00
Daniel Livingston
444762114c Initial somewhat working version 2022-03-19 15:11:13 -04:00
torzdf
860ccb91ac Suppress Tensorflow stderr custom error messages 2021-09-08 00:23:01 +01:00
torzdf
24888f61c3 Suppress ptxas error in GUI on Windows 2021-09-08 00:18:33 +01:00
torzdf
cf4b567cc6
Tensorflow 2.6 Support (#1182)
* lib.cli.launcher - Bump max tf version to 2.6

* Remove pathlib requirement

* Update requirements files

* Update setup.py

* bugfix - GUI: Supress errors when attempting to load previews in extract

* GUI: Suppress ptxas error messages for Windows
2021-09-05 02:25:12 +01:00
torzdf
6f1e6743df Bugfixes:
- Gui - Stats - Return empty dict on state file look up error
  - Gui - Last Session - Don't load saved project information when loading project from last session
  - Train - Set default coverage to 87.5%
2021-08-28 10:40:01 +00:00
torzdf
7e7640e557 Bugfix: Convert - Fix predicted mask 2021-08-11 23:58:34 +01:00
torzdf
ac22d40a91 extract: mask - Delete any mask from outside of frame boundaries 2021-07-22 13:04:29 +01:00
torzdf
eb96da0346 Add Misalignment Detection
- lib.align.AlignedFace - Add average_distance property (distance from mean_face)
  - tools.manual - Add misaligned Faces filter
  - tools.sort - Add sort by distance (misaligned sort)Add "Misaligned Faces" filter to manual tool
2021-06-08 19:30:28 +01:00
torzdf
a26bc50891 Manual Tool - Update to support masks with different centering 2021-05-31 18:58:34 +01:00
torzdf
18e3139275 Updates
- Change shortcut key to M
  - Slight optimizations
  - GUI support for mask toggling
2021-05-28 19:41:11 +01:00
torzdf
8c87ac52b4 Update Travis Tests for optimizers 2021-05-25 13:12:18 +00:00
torzdf
4c1631b58e lib.model - Add AdaBelief Optimizer 2021-05-25 13:11:35 +01:00
torzdf
6ee896d175 lib.gui.stats - Read loss names from model config output rather than state file 2021-05-23 16:10:42 +01:00
torzdf
3d914ee382 bugfix: Convert - don't error if no mask is selected 2021-05-21 10:52:39 +01:00
torzdf
00544f432a Bugfix: Collect mask correctly in training and convert 2021-05-20 13:39:16 +01:00
torzdf
0526da38a0 lib.align.pose.offset - Add legacy parameter to dict 2021-05-20 11:19:50 +01:00
torzdf
6c439944cd Masks updates
- Add head centering support to training
  - Update helptext/tooltips
2021-05-17 18:20:08 +01:00
torzdf
58cedef281 bugfix: lib.convert - Correctly pass mask centering to mask plugin 2021-05-17 12:06:18 +01:00
torzdf
4f48e1c25b bugfix: Training, don't error on loading extracted faces without stored_centering 2021-05-16 16:11:02 +01:00
torzdf
40798fd27f core mask updates:
- Add support for different mask centering
  - Update legacy alignments to store mask centering
  - Bugfix: lib.image ImageIO. Ensure unique queues are created (fixes mask tool when Face is input and an output folder is provided)
2021-05-16 13:24:00 +01:00
torzdf
ecd17d4ba7 Bugfix: Training Generator - Output image name with NoneType errors 2021-05-14 11:59:00 +01:00
torzdf
3835f3959d bugfix: augmentation: Use np.any for truth value 2021-05-14 01:54:26 +01:00
torzdf
add2d10315 Bugfix: Augmentation - Correctly calculate Clahe chance 2021-05-14 01:46:57 +01:00
torzdf
38d77f5583 bugfix: Training Generator - Catch Nonetypes when loading images 2021-05-14 00:29:27 +01:00
torzdf
cc5bc8743f GUI - Plugin preset support
- Add ability to load and save presets for all existing faceswap plugins
  - Add phaze-a presets
2021-05-08 13:31:49 +01:00
torzdf
3301c1ee29 Bugfix - Training - Fix eye/mouth multiplier for some models 2021-05-06 11:02:15 +00:00
torzdf
29beb7a23d bugfix - Fix race condition when WTL is selected 2021-05-05 16:40:20 +00:00
torzdf
51705fadb0 lib.model.losses_tf - Add multiplier bug catching code 2021-05-05 16:47:26 +01:00
torzdf
3092d1421c GUI Bugfix - Handle underscores in config options correctly 2021-05-01 11:09:14 +01:00
torzdf
e0a98e9b69 Training bugfixes:
- lib.training.generator
      - Fix duplicate "legacy faceset" warning
      - Fix missing mask error message
  - gui: Fix bug in live stats when resuming an old session
2021-04-20 13:56:03 +01:00
torzdf
c900036a4e GUI - Analysis Bugfix
- Get correct length of loss labels when carrying over raw data
2021-04-15 18:17:05 +01:00
torzdf
094ea338f1 GUI - Bugfixes
- Swallow OSErrors when failing to load preview image
  - Fix event_reader mapping for model output to loss names
  - stats - Ensure that _tb_logs exists prior to calling stop training
2021-04-14 15:28:57 +01:00
torzdf
a9bc9fb114 GUI - Wrapper - bugfix
- Only reset session data at the end of a training session
2021-04-13 11:57:18 +01:00
torzdf
510b8ba051 gui.analysis.event_reader - bugfix
- return empty when no timestamps in cache
2021-04-13 01:00:37 +01:00
torzdf
25d1f1b159 gui.analysis - Fixups
- Refactor stats and analysis code
  - Fix rollover data bug on live training
  - Update documentation
2021-04-13 00:10:29 +01:00
torzdf
e682d0c467 lib.gui.stats - Add debug code to catch stats graphing bug 2021-04-08 14:36:39 +01:00
torzdf
5ac15f68d2 gui - Console - Only strip trailing new lines from output 2021-04-03 12:53:47 +01:00
torzdf
51b316ee6c lib.training.generator - Catch mismatched image sizes in training folder 2021-04-03 11:31:58 +01:00
torzdf
26e9e20922 lib.training.generator - Add debug error code 2021-04-02 22:05:52 +01:00
torzdf
616bd7c500 GUI fixes
- Split themes and styles to own module
  - Fix border on console window
  - variable name updates to make pep8 happy
2021-03-25 19:29:29 +00:00
torzdf
5f598ca825 Locales Update 2021-03-22 18:41:55 +00:00
torzdf
a49831fb26 GUI - Add theme support to console 2021-03-22 14:45:18 +00:00
torzdf
29cfdaad46 Training startup updates
- Remove alignments path option from cli
  - Restructure training_data.py
  - Load alignment data from PNG header during first epoch
  - lib.image.read_image_batch - Add option to return metadata
  - lib.utils.get_image_paths - Add option for explicit extension
  - plugins.train.trainer._base - remove pre-cache alignments code
  - scripts.train - Check first image in training folders for metadata
  - Documentation
2021-03-21 23:29:31 +00:00
torzdf
6872173d7e GUI - Add missing file 2021-03-20 14:08:53 +00:00
torzdf
a49f810e7c GUI - Make tooltips themeable 2021-03-20 14:08:07 +00:00
torzdf
46cb53e6f0 Training updates:
- Expose optimizer epsilon param
  - Add NaN protection
GUI - slider - add support for discreet values
2021-03-19 18:22:35 +00:00
torzdf
a656552204 Minor Updates
- GUI - Tweaks
    - Fix Analysis Slider background color
    - More granular theming controls
    - Standardize comboboxes and scrollbars
  - plugins.train.model._base
      - Catch more model corruption errors
2021-03-18 17:43:56 +00:00
mark-gargan
d0e9990870
Adding some defensive code to better identify which image is causing … (#1133)
* Adding some defensive code to better identify which image is causing issue when permission error is encountered.
2021-03-18 12:13:12 +00:00
torzdf
11009bf237 GUI Bugfix - Stop settings colors leaking into analysis. 2021-03-16 23:20:19 +00:00
torzdf
cff0f5698b GUI updates
- Control panel fixes
  - Basic themeing support
  - Fix tools to display more natively
  - Give all config items a group
2021-03-16 19:46:14 +00:00
torzdf
46edd5e5ae GUI fix - Make grey slider backgrounds go away 2021-03-15 00:38:18 +00:00
torzdf
1c0d112588 Minor GUI Updates
- Fix spacing between buttons on Control Panels
  - Configurable background on Control Panels
  - Fix background color of TreeView menu in settings pop up
  - Change TreeView selected item highlight color
  - Change console background color to match control panel
  - Fix unfilled color in control panel background
2021-03-14 23:16:08 +00:00
torzdf
7115ad3550 GUI - Minor Updates
- Add style support to control panel
  - Theme settings to red
  - Use standard tk buttons
  - Fix some graphical glitches
2021-03-14 20:27:04 +00:00
torzdf
5eea32833d GUI - Make groups collapsible 2021-03-14 02:50:06 +00:00