If you have heard about LoRA models for Stable Diffusion at some point, you might have also heard something about LyCORIS. What is it exactly, how does it differ from regular LoRAs if at all, and how can you make use of it in Stable Diffusion Automatic1111 WebUI? We’re going to answer all these questions here.
The article was updated for the version 1.5.0 and all versions above, and now includes the new, correct way to import LyCORIS models!
Use the table of contents below to quickly jump to the sections of this article that might interest you!
Before you learn about LyCORIS, you might first be interested in this: How to Use LoRA Models with Stable Diffusion WebUI – Our Quick Tutorial
What Is LyCORIS And How Is It Different From LoRA?
LyCORIS actually stands for Lora beYond Conventional methods, Other Rank adaptation Implementations for Stable diffusion. This is a rather lengthy name, isn’t it? Here is exactly what it refers to.
LyCORIS, according to the official KohakuBlueleaf LyCORIS GitHub repository, is a whole project with the goal of exploring different ways of parameter-efficient Stable Diffusion fine-tuning via researched and implementation of different fine-tuning algorithms. If this doesn’t really tell you much, read on!
In other words, LyCORIS is a whole set of a few different fine-tuning methods for Stable Diffusion. Fine-tuning, as a reminder, in this context means altering a Stable Diffusion based model using different means to be able to achieve certain generation styles that might not be easy to achieve otherwise. So…
Is LyCORIS the same as LoRA? Under the name LyCORIS, hide a few different ways of fine-tuning Stable Diffusion to your specific needs. These include the ever-popular conventional LoRAs – so the nifty low-rank adaptation models (and their improved-on versions) that we have already covered here on techtactician.com in great detail.
The types of fine-tuning methods that LyCORIS includes are, to name a few:
- LoRA/LoCON (the Conventional LoRA method and its improved version).
- LoHa (LoRA with Hadamard Product representation).
- LoKR (LoRA with Kronecker Product representation).
- DyLoRA (LoRA Using Dynamic Search-Free Low Rank Adaptation).
Bet you didn’t know about some of these!
LyCORIS vs. LoRA? - All in all, LyCORIS can be defined as a whole family of fine-tuning methods that includes the "traditional" conventional LoRA models, and builds upon their initial concept to further optimize both the training and inference process of said fine-tunings.
You can find the full set of LyCORIS fine-tuning methods with their short descriptions in the official LyCORIS GitHub repository that we have already mentioned.
Check out also: How To Train Own Stable Diffusion LoRA Models – Full Tutorial!
So, Can I use LyCORIS Models Just Like LoRA?
Yes! As you will see in a while, since the Stable Diffusion WebUI version 1.5.0, you can import, invoke and all the models from the LyCORIS family just as you would use regular LoRA models!
If you want to know how exactly to do this, scroll down, or get to our original article on LoRA model usage with the Automatic1111 WebUI!
Is The LyCORIS Extension Still Needed For The Automatic1111 WebUI?
You will be pleased to learn that the a1111-sd-webui-lycoris extension is no longer needed for the Automatic1111 WebUI to recognize the LyCORIS type models.
From the update 1.5.0 going forward, you don’t need to install the sd-webui-lycoris extension anymore, as the WebUI now has native support for LyCORIS models.
The WebUI is now able to automatically make use of any LyCORIS model you throw at it without you having to import any additional extension plugins, which is an update we’ve been waiting for for a long time now!
Here is exactly how to use the LyCORIS models with the newer versions of the Stable Diffusion WebUI.
How To Use LyCORIS Models In Stable Diffusion WebUI
How do you use LyCORIS models in the Stable Diffusion Automatic1111 WebUI? Here are the exact steps!
Step 1: Download a LyCORIS Model And Import It
First, download a LyCORIS model that you want to use, and put it in the \stable-diffusion-webui\models\LoRA directory. Note that in the Stable Diffusion WebUI LoRA models and LyCORIS models are stored in the exact same directory since the version 1.5.0.
You can find lots of different LyCORIS models over on Civit.ai using their neat library filters, if you don’t have any of these downloaded yet.
Step 2: Trigger The LyCORIS Model During Image Generation
Once you click the Lora tab, the library with all your LoRA and LyCORIS models should show up.
Now you’ll have to click on the model tile that you want to use during generation. A string looking something like this should be added to your main prompt: <lora:yourmodelname:1>
Step 3: Keep In Mind The LyCORIS/LoRA Weight and Trigger Words
Similar to LoRA models, the number in the end of the LyCORIS activation string (number 1 in our example above), indicates the “weight” of the LyCORIS model in the prompt.
1 means that the LyCORIS model is in full effect, which might be too much in certain cases, lower values like 0.6 would give the LyCORIS model less impact on the generation, while a higher value like 1.4 would further amplify the model’s final impact.
Remember that setting the LyCORIS weights too high in most cases will introduce unwanted distortion and artifacts into your image, just as it would with regular LoRA models.
Very Important Note: Some LyCORIS models, just like some LoRA models, may require certain “trigger words/keywords” to be used alongside the model activation string you’ve seen above.
The information about whether or not a model needs a trigger word/words to function optimally is always available somewhere in the model description on the site you download your LyCORIS/LoRA model from. The only thing you need to do to use a trigger word/keyword is to include it somewhere in your main prompt!
If you’ve done everything correctly, you now should be able to enjoy using all the LyCORIS models that you’ve downloaded in your generations!
Last important note: LyCORIS models, just like conventional LoRA models, when trained in a certain style or based on a certain checkpoint might not work well with all Stable Diffusion models/checkpoints you attempt to use them on. Choose the base SD model you’re using your LyCORIS model on accordingly to the output image style you want to achieve!
You might also like: 7 Benefits Of Upgrading Your GPU Today – Is It Worth It? – Why Should You?
What To Do If Your LyCORIS Models Aren’t Working?
If your LyCORIS models are not showing up in the LoRA tab, make sure that you have the LyCORIS model file imported into the correct folder (the \stable-diffusion-webui\models\LyCORIS directory), and that you have refreshed the LoRA library using the refresh button in the Lora tab.
If your LyCORIS models are showing up in the tab and you have imported them into the prompt correctly, make sure that your imported model in your prompt looks like this “<lora:yourmodelname:1>”, rather than like this: “<lyco:yourmodelname:1>”.
The latter is the old way of invoking LyCORIS models which was used in older versions of the Automatic1111 WebUI (pre-1.5.0), and it won’t work correctly with the newer versions.
LyCORIS vs. LoRA In Stable Diffusion
As we’ve already established, LyCORIS is the general name for a group of quite a few different researched methods of fine-tuning Stable Diffusion. This group, according to the official LyCORIS GitHub repository, does include the improved version of the conventional LoRAs you might have heard about before.
So, if we were to compare the basic LoRA models to other newer and extended methods of fine-tuning Stable Diffusion via upgraded LyCORIS based forms of low-rank adaptation, we would have to compare each of the other methods directly with conventional LoRAs in terms of training efficiency, final model size, output qualities and more.
These kinds of experimental comparisons have been already made to an extent, for example between the Hadamard product LoRAs (LoHAs) and Conventional LoRAs.
All the official comparisons are available here in form of grids of images generated using different methods of fine-tuning, different amounts of steps when training the LyCORIS models and different LyCORIS model prompt weights.
While the general training speed and number of required training steps for good results differ between different LyCORIS methods, the quality difference between the images generated with optimal settings is generally rather small. Check the available examples out if you want to see for yourself!
How To Tell A Conventional LoRA Model From a LyCORIS model?
There is a chance that you’ve just downloaded a ton of different models of both traditional LoRA type and the newer LyCORIS (LoCon/LoHA) type, both of which are readily available at civit.ai. Now you might ask: how do I tell one from the other, when all the files look the same after download?
Fear not, as there is a quick solution for this even if you’ve mixed up your traditional LoRA and LyCORIS (LoCon/LoHA) models together and can’t tell one from the other. There is a neat Automatic1111 Stable Diffusion WebUI extension that can do this (and more) for you!
The extension we’re talking about is the Civitai Helper 2 / ModelInfo Helper extension.
After installing it either from the Automatic1111 WebUI extension menu, or sourcing it from its official GitHub repository and then restarting the WebUI, it will be ready to go.
The Civitai Helper 2 / ModelInfo Helper extension allows you to view more info about your imported models right in the WebUI, and it also is able to generate a .json file with all the other model details next to each model file in the appropriate Stable Diffusion models directory.
With the help of this extension you will not only be able to tell which model is a standard LoRA, and which one is a LyCORIS based model, but also automatically generate model card previews, add the model’s specific trigger words to your prompts and more. Pretty neat!
Can You Train Your Own LyCORIS Models Locally?
Yes, and the easiest method of doing that is really similar to training your very own LoRA models. As this process is kinda lengthy, we won’t get into much detail in this article. In the meantime, you can check out our simplified LoRA training guide for beginners.
The LyCORIS model training process differs only in a few places from regular LoRA training. We really recommend you to learn how to train your own LoRAs first – it’s really simple!
Check this out! – How To Train Own Stable Diffusion LoRA Models – Full Tutorial!
