If you have heard about LoRA models for Stable Diffusion at some point, you might have also heard something about LyCORIS. What is it exactly, how does it differ from regular LoRAs if at all, and how can you make use of it in Stable Diffusion Automatic1111 WebUI? We’re going to answer all these questions here.
Use the table of contents below to quickly jump to the sections of this article that might interest you!
- What Is LyCORIS And How Does It Differ From LoRA?
- “This is LyCORIS (LoCon/LoHA) model…” – Civit.ai
- How To Install The LyCORIS Extension For The WebUI?
- How To Use LyCORIS Models In Stable Diffusion WebUI?
- LyCORIS vs. LoRA In Stable Diffusion
- How To Tell A Conventional LoRA Model From a LyCORIS model?
- Can You Train Your Own LyCORIS Models Locally?
Before you learn about LyCORIS, you might first be interested in this: How to Use LoRA Models with Stable Diffusion WebUI – Our Quick Tutorial
What Is LyCORIS And How Does It Differ From LoRA?
LyCORIS actually stands for Lora beYond Conventional methods, Other Rank adaptation Implementations for Stable diffusion. This is a rather lengthy name, isn’t it? Let’s see exactly what it refers to.
Well, LyCORIS, according to the official KohakuBlueleaf LyCORIS GitHub repository, is a whole project with the goal of exploring different ways of parameter-efficient Stable Diffusion fine-tuning via researched and implementation of different fine-tuning algorithms. If this doesn’t really tell you much, read on!
In other words, LyCORIS is a whole set of a few different fine-tuning methods for Stable Diffusion. Fine-tuning in this context means altering a Stable Diffusion based model using different means to be able to achieve certain generation styles that might not be easy to achieve otherwise, if you didn’t know that already. So…
Under the name LyCORIS, hide a few different ways of fine-tuning Stable Diffusion to your specific needs. These include the ever-popular conventional LoRAs – so the nifty low-rank adaptation models (and their improved-on versions) that we have already covered here on techtactician.com in great detail.
The types of fine-tuning methods that LyCORIS includes are, to name a few:
- LoRA/LoCON (the Conventional LoRA method and its improved version).
- LoHa (LoRA with Hadamard Product representation).
- LoKR (LoRA with Kronecker Product representation).
- DyLoRA (LoRA Using Dynamic Search-Free Low Rank Adaptation).
Bet you didn’t know about some of these!
All in all, LyCORIS can be defined as a whole family of fine-tuning methods that includes the "traditional" conventional LoRA models, and builds upon their initial concept to further optimize both the training and inference process of said fine-tunings.
You can find the full set of LyCORIS fine-tuning methods with their short descriptions in the official LyCORIS GitHub repository that we have already mentioned.
Check out also: How To Train Own Stable Diffusion LoRA Models – Full Tutorial!
“This is LyCORIS (LoCon/LoHA) model…” – Civit.ai
When browsing through hundreds of different free LyCORIS based fine-tunings on the great web resource for Stable Diffusion downloads that is Civit.ai, you might come across this little message next to a LyCORIS model download button.
"This is a LyCORIS (LoCon/LoHA) model, and requires an additional extension in Automatic 1111 to work."What this message is trying to tell you, is that to use LyCORIS based models in the Stable Diffusion WebUI, you’ll need an official extension for the Automatic1111 WebUI so that the software can recognize and properly utilize LyCORIS models in your generations.
Keep on reading to learn how exactly to use LyCORIS models within the WebUI – you can learn how to do this in no more than 5 minutes total!
You might also like: RVC WebUI How To – Make AI Song Covers in Minutes! (Voice Conversion Guide)
How To Install The LyCORIS Extension For The WebUI?
If you want to use LyCORIS models with the Automatic1111 Stable Diffusion WebUI, you’re in luck, as the process is extremely easy and it has only one extra step when compared with utilizing conventional LoRA models within the WebUI.
The one extra step needed for using LyCORIS based models in the WebUI, is downloading and installing the a1111-sd-webui-lycoris LyCORIS extension. We’ll now show you exactly how to do that.
- First, open up your Automatic1111 Stable Diffusion WebUI and navigate to the extension tab.
- Now, copy the a1111-sd-webui-lycoris extension link to your clipboard – https://github.com/KohakuBlueleaf/a1111-sd-webui-lycoris.git
- Then, in the “from url” tab, enter the link you just copied and click on the install button.
- After a WebUI restart the LyCORIS extension should be installed, and the WebUI should now be able to utilize LyCORIS models.
You can also install the extension by using the extension search feature in the WebUI, or by manually downloading the extension files from its GitHub repository and moving them into the WebUI extensions folder and restarting the WebUI.
To learn how exactly to use LyCORIS models in the Stable Diffusion WebUI in the correct way, head over to the next paragraph.
How To Use LyCORIS Models In Stable Diffusion WebUI?
Here are the exact steps that you need to take to use LyCORIS models in the Automatic1111 WebUI:
Step 1: Pick a LyCORIS Model
First, download a LyCORIS model that you want to use, and put it in the \stable-diffusion-webui\models\LyCORIS directory. Note that in the Stable Diffusion WebUI LoRA models and LyCORIS models are stored in two different directories.
You can find lots of different LyCORIS models over on Civit.ai using their neat library filters, if you don’t have any of these downloaded yet.
Step 2: Install The LyCORIS Extension For The WebUI
Next, make sure that you have the LyCORIS extension for the Automatic1111 WebUI installed. Without it, the WebUI won’t be able to run LyCORIS models correctly. If you don’t know how to do that yet, scroll up! After installing the extension, restart the WebUI.
Step 3: Locate And Trigger The LyCORIS Model
In the main WebUI window, click the Image Style Library button located under the Generate button (shown on the image below).
Once you click the Image Style Library button, the library with a few tabs should show up. If you have installed the LyCORIS extension properly, a LyCORIS model tab should show up here, right beside the LoRA model tab.
Now you’ll have to click on the LyCORIS model tile that you want to use during generation. A string looking something like this should be added to your main prompt: <lyco:yourloramodel:1>
Step 4: Keep In Mind The LyCORIS/LoRA Weight and Trigger Words
Similar to LoRA models, the number in the end of the LyCORIS activation string (number 1 in our example above), indicates the “weight” of the LyCORIS model in the prompt.
1 means that the LyCORIS model is in full effect, which might be too much in certain cases, lower values like 0.6 would give the LyCORIS model less impact on the generation, while a higher value like 1.4 would further amplify the model’s final impact.
Remember that setting the LyCORIS weights too high in most cases will introduce unwanted distortion into your image, just as it would with regular LoRA models.
Very Important Note: Some LyCORIS models, just like some LoRA models, may require certain “trigger words/keywords” to be used alongside the model activation string you’ve seen above.
The information about whether or not a model needs a trigger word/words to function optimally is always available somewhere in the model description on the site you download your LyCORIS/LoRA model from. The only thing you need to do to use a trigger word/keyword is to include it somewhere in your main prompt!
If you have installed the LyCORIS extension and followed the instructions correctly, you now should be able to enjoy using all the LyCORIS models that you’ve downloaded in your generations!
Last important note: LyCORIS models, just like conventional LoRA models, when trained in a certain style or based on a certain checkpoint might not work well with all Stable Diffusion models/checkpoints you attempt to use them on. Choose the base SD model you’re using your LyCORIS model on accordingly to the output image style you want to achieve!
You might also like: 7 Benefits Of Upgrading Your GPU Today – Is It Worth It? – Why Should You?
LyCORIS vs. LoRA In Stable Diffusion
As we’ve already established, LyCORIS is the general name for a group of quite a few different researched methods of fine-tuning Stable Diffusion. This group, according to the official LyCORIS GitHub repository, does include the improved version of the conventional LoRAs you might have heard about before.
So, if we were to compare the basic LoRA models to other newer and extended methods of fine-tuning Stable Diffusion via upgraded LyCORIS based forms of low-rank adaptation, we would have to compare each of the other methods directly with conventional LoRAs in terms of training efficiency, final model size, output qualities and more.
These kinds of experimental comparisons have been already made to an extent, for example between the Hadamard product LoRAs (LoHAs) and Conventional LoRAs.
All the official comparisons are available here in form of grids of images generated using different methods of fine-tuning, different amounts of steps when training the LyCORIS models and different LyCORIS model prompt weights.
While the general training speed and number of required training steps for good results differ between different LyCORIS methods, the quality difference between the images generated with optimal settings is generally rather small. Check the available examples out if you want to see for yourself!
How To Tell A Conventional LoRA Model From a LyCORIS model?
There is a chance that you’ve just downloaded a ton of different models of both traditional LoRA type and the newer LyCORIS (LoCon/LoHA) type, both of which are readily available at civit.ai. Now you might ask: how do I tell one from the other, when all the files look the same after download?
Fear not, as there is a quick solution for this even if you’ve mixed up your traditional LoRA and LyCORIS (LoCon/LoHA) models together and can’t tell one from the other. There is a neat Automatic1111 Stable Diffusion WebUI extension that can do this (and more) for you!
The extension we’re talking about is the Civitai Helper 2 / ModelInfo Helper extension.
After installing it either from the Automatic1111 WebUI extension menu, or sourcing it from its official GitHub repository and then restarting the WebUI, it will be ready to go.
The Civitai Helper 2 / ModelInfo Helper extension allows you to view more info about your imported models right in the WebUI, and it also is able to generate a .json file with all the other model details next to each model file in the appropriate Stable Diffusion models directory.
With the help of this extension you will not only be able to tell which model is a standard LoRA, and which one is a LyCORIS based model, but also automatically generate model card previews, add the model’s specific trigger words to your prompts and more. Pretty neat!
Can You Train Your Own LyCORIS Models Locally?
Yes, and the easiest method of doing that is really similar to training your very own LoRA models. As this process is kinda lengthy, we won’t get into much detail in this article. In the meantime, you can check out our simplified LoRA training guide for beginners.
The LyCORIS model training process differs only in a few places from regular LoRA training. We really recommend you to learn how to train your own LoRAs first – it’s really simple!
Check out also: How To Train Own Stable Diffusion LoRA Models – Full Tutorial!
