A free and open-source inpainting tool powered by SOTA AI model.
## Features
- Completely free and open-source
- Fully self-hosted
- Classical image inpainting algorithm powered by [cv2](https://docs.opencv.org/3.4/df/d3d/tutorial_py_inpainting.html)
- Multiple SOTA AI models
1. [LaMa](https://github.com/saic-mdal/lama)
1. [LDM](https://github.com/CompVis/latent-diffusion)
1. [ZITS](https://github.com/DQiaole/ZITS_inpainting)
1. [MAT](https://github.com/fenglinglwb/MAT)
1. [FcF](https://github.com/SHI-Labs/FcF-Inpainting)
1. [SD1.4](https://github.com/CompVis/stable-diffusion)
- Support CPU & GPU
- Various inpainting [strategy](#inpainting-strategy)
- Run as a desktop APP
## Usage
| Usage | Before | After |
| ---------------------- | --------------------------------------------- | -------------------------------------------------------------- |
| Remove unwanted things |  |  |
| Remove unwanted person |  |  |
| Remove Text |  |  |
| Remove watermark |  |  |
| Fix old photo |  |  |
| Text Driven Inpainting |  | Prompt: a fox sitting on a bench  |
## Quick Start
```bash
pip install lama-cleaner
# Model will be downloaded automatically
lama-cleaner --model=lama --device=cpu --port=8080
# Lama Cleaner is now running at http://localhost:8080
```
Available arguments:
| Name | Description | Default |
| ----------------- | -------------------------------------------------------------------------------------------------------- | -------- |
| --model | lama/ldm/zits/mat/fcf/sd. See details in [Inpaint Model](#inpainting-model) | lama |
| --hf_access_token | stable-diffusion(sd) model need huggingface access token https://huggingface.co/docs/hub/security-tokens | |
| --device | cuda or cpu | cuda |
| --port | Port for backend flask web server | 8080 |
| --gui | Launch lama-cleaner as a desktop application | |
| --gui_size | Set the window size for the application | 1200 900 |
| --input | Path to image you want to load by default | None |
| --debug | Enable debug mode for flask web server | |
## Inpainting Model
| Model | Description | Config |
| ----- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| cv2 | :+1: No GPU is required, and for simple backgrounds, the results may even be better than AI models. | |
| LaMa | :+1: Generalizes well on high resolutions(~2k) | |
| LDM | :+1: Possible to get better and more detail result :+1: The balance of time and quality can be achieved by adjusting `steps` :neutral_face: Slower than GAN model :neutral_face: Need more GPU memory | `Steps`: You can get better result with large steps, but it will be more time-consuming `Sampler`: ddim or [plms](https://arxiv.org/abs/2202.09778). In general plms can get [better results](https://github.com/Sanster/lama-cleaner/releases/tag/0.13.0) with fewer steps |
| ZITS | :+1: Better holistic structures compared with previous methods :neutral_face: Wireframe module is **very** slow on CPU | `Wireframe`: Enable edge and line detect |
| MAT | TODO | |
| FcF | :+1: Better structure and texture generation :neutral_face: Only support fixed size (512x512) input | |
| SD1.4 | :+1: SOTA text-to-image diffusion model | |
### LaMa vs LDM
| Original Image | LaMa | LDM |
| ----------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------- |
|  |  |  |
### LaMa vs ZITS
| Original Image | ZITS | LaMa |
| ---------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------- |
|  |  |  |
Image is from [ZITS](https://github.com/DQiaole/ZITS_inpainting) paper. I didn't find a good example to show the advantages of ZITS and let me know if you have a good example. There can also be possible problems with my code, if you find them, please let me know too!
### LaMa vs FcF
| Original Image | Lama | FcF |
| ----------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------- |
|  |  |  |
## Inpainting Strategy
Lama Cleaner provides three ways to run inpainting model on images, you can change it in the settings dialog.
| Strategy | Description | VRAM |
| ------------ | ---------------------------------------------------------------------------------------------------------------------------------------------- | -------------------- |
| **Original** | Use the resolution of the original image | :tada: |
| **Resize** | Resize the image to a smaller size before inpainting. Lama Cleaner will make sure that the area of the image outside the mask is not degraded. | :tada: :tada: |
| **Crop** | Crop masking area from the original image to do inpainting | :tada: :tada: :tada: |
## Download Model Mannually
If you have problems downloading the model automatically when lama-cleaner start,
you can download it manually. By default lama-cleaner will load model from `TORCH_HOME=~/.cache/torch/hub/checkpoints/`,
you can set `TORCH_HOME` to other folder and put the models there.
- Github:
- [LaMa](https://github.com/Sanster/models/releases/tag/add_big_lama)
- [LDM](https://github.com/Sanster/models/releases/tag/add_ldm)
- [ZITS](https://github.com/Sanster/models/releases/tag/add_zits)
- [MAT](https://github.com/Sanster/models/releases/tag/add_mat)
- [FcF](https://github.com/Sanster/models/releases/tag/add_fcf)
- Baidu:
- https://pan.baidu.com/s/1vUd3BVqIpK6e8N_EA_ZJfw
- passward: flsu
## Development
Only needed if you plan to modify the frontend and recompile yourself.
### Frontend
Frontend code are modified from [cleanup.pictures](https://github.com/initml/cleanup.pictures), You can experience their
great online services [here](https://cleanup.pictures/).
- Install dependencies:`cd lama_cleaner/app/ && yarn`
- Start development server: `yarn start`
- Build: `yarn build`
## Docker
Run within a Docker container. Set the `CACHE_DIR` to models location path. Optionally add a `-d` option to
the `docker run` command below to run as a daemon.
### Build Docker image
```
docker build -f Dockerfile -t lamacleaner .
```
### Run Docker (cpu)
```
docker run -p 8080:8080 -e CACHE_DIR=/app/models -v $(pwd)/models:/app/models -v $(pwd):/app --rm lamacleaner python3 main.py --device=cpu --port=8080
```
### Run Docker (gpu)
```
docker run --gpus all -p 8080:8080 -e CACHE_DIR=/app/models -v $(pwd)/models:/app/models -v $(pwd):/app --rm lamacleaner python3 main.py --device=cuda --port=8080
```
Then open [http://localhost:8080](http://localhost:8080)
