[PS3] (1.2 Update) 4K Upscaled Persona 5 Royal Bustups Mod

Share, discover, or contribute to mods for Persona 5 for PS3 and PS4.
User avatar
shst
Posts: 13
Joined: Tue Nov 03, 2020 10:46 am
Has thanked: 1 time
Been thanked: 2 times

Re: 4K Upscaled Persona 5 Royal Bustups Mod

Post by shst » Fri Dec 04, 2020 8:34 am

LagBeam wrote:
Fri Dec 04, 2020 12:48 am
I've done (manually which sucks) a decent chunk of bustups with different upscaling technique as per my posts above.
Haven't looked at your files before now. Results with RealSR look fine, but are still on a blurrier side, which is especially noticeable on outlines. My solution uses ESRGAN with mostly the same models as Full 4K Upscale, which provides superb quality.
LagBeam wrote:
Fri Dec 04, 2020 12:48 am
Unlike CPU (2 posts above) I've used GPU (Radeon 5700XT) so I can actually assist on upscaling (won't take 2+ hours to do a single set of bustup)
On original hardware it was even worse, about 4 hours for a single image when using ESRGAN fork in a mode that correctly preserves PNG transparency. I've rented a VPS for a month with a slightly better specs than my PC (sans the GPU), and with rebuilt PyTorch with Intel MKL support and other CPU optimizations it takes about 80-90 minutes to process a full bustup (768x768 image). As I'm writing this post, I've been running it for 72 hours, and it's halfway done with Ryuji. It is possible that I might obtain a CUDA-compatible GPU by the end of the month, if it won't be done by that moment.

In any case, my approach is pretty brute - take all GNF images, convert them, delete the blank images (which are just small white squares), process the rest, then just do a bulk replace of them all within this mod, without any changes to bustup tables themselves. Just to see if results would be bearable.

LagBeam
Posts: 17
Joined: Tue Aug 04, 2020 6:03 am
Location: Russia
Has thanked: 5 times
Been thanked: 1 time

Re: 4K Upscaled Persona 5 Royal Bustups Mod

Post by LagBeam » Mon Dec 07, 2020 2:53 pm

alright, I hope you've checked those bustups during animations and there are no artificial seams (which happened to me unless I rebuild full portraits). Hope everything works out for you. I'm still up for sharing the workload.
Also your image is 3k unlike "OG" mod here and my jabs at it (1.5k)
EDIT: Ran your models count. Why the BC1 models were selected? I think they actually detrimental to visual detail - ran fatality models with and without BC1 ones - buttons are noticeably sharper and detailed - PS4 bustups should (emphasis on should) have much better compression (bc3/dxt5 or even bc5 - i don't remember) for the entire dataset unlike PS3 levels of compression (especially if we are making a 4k mod, not retrofitting these bustups) (also that image did not take 40 minutes to process with 4 or 2 models on my puny hardware (GT1030 for CUDA))
Also why not consider using Cupscale and it's (for instance, can just use OCNN2NCNN) ability to utilize Vulkan? - I've converted said models and tried running test samples. Sure its a work in progress but it usability in case of bustups and their pieces depends on how accurately CUDA can process overlaps and seams. So far my results are awful in terms of seams-removal, making it easier for me to just rebuild portraits as I was doing it before.

Yet Another Update: after some trials I am considering stopping with only last 2 models in your suggested workflow (Futaba's iris retains 5 sploches in this case, unlike inclusion of any BC1 models). So i will be using those 2 "Fatality" models in my personal version (RealSR is indeed blurrier because of de-noising and no ability to turn down the effect except remove it completely)

User avatar
shst
Posts: 13
Joined: Tue Nov 03, 2020 10:46 am
Has thanked: 1 time
Been thanked: 2 times

Re: 4K Upscaled Persona 5 Royal Bustups Mod

Post by shst » Tue Dec 08, 2020 1:20 pm

LagBeam wrote:
Mon Dec 07, 2020 2:53 pm
alright, I hope you've checked those bustups during animations and there are no artificial seams (which happened to me unless I rebuild full portraits). Hope everything works out for you. I'm still up for sharing the workload.
Sadly, there are, but this far it all seems to be somewhat fixable.
LagBeam wrote:
Mon Dec 07, 2020 2:53 pm
Also your image is 3k unlike "OG" mod here and my jabs at it (1.5k)
Yeah, I didn't downscale it, and 4x models upscale images 4 times, so 768 -> 3k.
LagBeam wrote:
Mon Dec 07, 2020 2:53 pm
Why the BC1 models were selected?
They were mentioned in readme of original 4K upscale mod, though some models were not found, so I replaced them with ones that match names the closest. And I think I saw a little bit of something that looks like compression artifacts in P5R bustups on outlines, so I decided to play it safe.
LagBeam wrote:
Mon Dec 07, 2020 2:53 pm
ran fatality models with and without BC1 ones - buttons are noticeably sharper and detailed
...
Futaba's iris retains 5 sploches in this case, unlike inclusion of any BC1 models
Could you upload an example? I'm quite curious how that looks, and I don't have an access to a server or main PC right now.
LagBeam wrote:
Mon Dec 07, 2020 2:53 pm
Also why not consider using Cupscale and it's (for instance, can just use OCNN2NCNN) ability to utilize Vulkan?
I'm mainly a Linux user, and I didn't find native NCNN version of ESRGAN. I tried to use its ready-made version from Cupscale binaries under Wine, but it seems that it only works with 4x models. and trying to force it to use 1x models results in absolute mess of an image with all the elemets getting duplicated several times (sadly, I didn't save an example). I'm not sure about its PNG transparency support, either.

LagBeam
Posts: 17
Joined: Tue Aug 04, 2020 6:03 am
Location: Russia
Has thanked: 5 times
Been thanked: 1 time

Re: 4K Upscaled Persona 5 Royal Bustups Mod

Post by LagBeam » Tue Dec 08, 2020 3:43 pm

Cupscale has decent transparency support, I had troubles running with python (but fixed it)
upload of the image with slider - https://imgsli.com/MzI0Mzk
standalone image - http://www.mediafire.com/view/nvmzv7q1j ... s.png/file
I can agree that image may seem somewhere overdone but floats for me (IMO really looks closer to original 768p than over smoothened image after BC1 models)
with TTA mode (non-tta modes are noticeably noisy but match CUDA almost on point) on my hardware takes a hot minute for 2 models to be processed back-to-back (5-ish, didn't count)
EDIT: about animations - I hope there is proper automation way of doing it, otherwise its manual labor - I can re-do my 40~% of work with Fatality models but everything else needs work (recombine bustups etc. etc.)
EDIT 2: I'm starting to think about different models to use - I kinda don't like results of 2-step approach (namely caruncula (got from wikipedia)) but i still like it a bit more for reasons mention earlier. Just using upscaling model provides softer image than i would like to so combination seems necessary.

User avatar
shst
Posts: 13
Joined: Tue Nov 03, 2020 10:46 am
Has thanked: 1 time
Been thanked: 2 times

Re: 4K Upscaled Persona 5 Royal Bustups Mod

Post by shst » Wed Dec 09, 2020 2:27 am

I've actually done one example myself, on ESRGAN fork in CPU mode: https://files.catbox.moe/oqmvxv.png, with only the last two models (MKII and DeBlur). And it seems that my result is noticeably noisier than yours, which is strange if we used the exact same two models. Still, headphone button quality is better. Did you do any further processing?

EDIT: used waifu2x-cpp in noise removal mode, it returns smooth image with seemingly no details lost: https://de.catbox.moe/6yf39o.png. It doesn't seem to have any additional sharpness featured in your result, though, and I don't know if that's for better or worse.

LagBeam
Posts: 17
Joined: Tue Aug 04, 2020 6:03 am
Location: Russia
Has thanked: 5 times
Been thanked: 1 time

Re: 4K Upscaled Persona 5 Royal Bustups Mod

Post by LagBeam » Wed Dec 09, 2020 3:18 am

shst wrote:
Wed Dec 09, 2020 2:27 am
I've actually done one example myself, on ESRGAN fork in CPU mode: https://files.catbox.moe/oqmvxv.png, with only the last two models (MKII and DeBlur). And it seems that my result is noticeably noisier than yours, which is strange if we used the exact same two models. Still, headphone button quality is better. Did you do any further processing?
I used NCNN version by getting Cupscale and getting esrgan-ncnn processor - it has toggle for TTA mode, i did mine with that enabled (according to Waifu2x-GUI TTA mode if enabled, increases SNR by 0.15 (roughly) - in our case it provides MUCH cleaner output.
IF I disable TTA mode and do full 4 models processing - i get near identical result to your first posts in this thread.
To run NCNN version I took CUDA models and converted them (available in Cupscale)
Since esrgan-ncnn is effectively realsr-ncnn-vulkan (iirc it's a fork of realsr) - either one can be used (just gotta specify correct model and parameters)

User avatar
shst
Posts: 13
Joined: Tue Nov 03, 2020 10:46 am
Has thanked: 1 time
Been thanked: 2 times

Re: 4K Upscaled Persona 5 Royal Bustups Mod

Post by shst » Wed Dec 09, 2020 5:19 am

LagBeam wrote:
Wed Dec 09, 2020 3:18 am
Since esrgan-ncnn is effectively realsr-ncnn-vulkan (iirc it's a fork of realsr) - either one can be used (just gotta specify correct model and parameters)
Yeah, seems like Cupscale author credited authors of realsr-ncnn. It wasn't in readme when I initially tried it out. Might try again with it.

UPD: tried it out. Took 45 seconds to upscale and 7 minutes to deblur on my RX 460, unlike 4 hours on CPU with regular ESRGAN. In matters of noise it's the same as previous version, but outlines look significally worse, being both pixelated and blurry.

LagBeam
Posts: 17
Joined: Tue Aug 04, 2020 6:03 am
Location: Russia
Has thanked: 5 times
Been thanked: 1 time

Re: 4K Upscaled Persona 5 Royal Bustups Mod

Post by LagBeam » Wed Dec 09, 2020 6:31 am

shst wrote:
Wed Dec 09, 2020 5:19 am
UPD: tried it out. Took 45 seconds to upscale and 7 minutes to deblur on my RX 460, unlike 4 hours on CPU with regular ESRGAN. In matters of noise it's the same as previous version, but outlines look significally worse, being both pixelated and blurry.
Have you used it with TTA on or off? if off, then it should be indistinguishable.
EDIT: Checked the image - its 2 steps FAST (TTA-mode OFF), models as discussed.
TTA-enabled improves results, but "roughly" quadruples the time required. Also tile size could affect overall speed (less time shifting between tiles, I assume)
EDIT 2: In my case (R9 3950x, RX 5700XT, GT1030 (not used for NCNN-Vulkan)) - its ~16 minutes with TTA-on and tile-size set to Auto (0)
its ~12 minutes if TTA-on and tile-size adjusted to be the max possible value my GPU can handle

User avatar
shst
Posts: 13
Joined: Tue Nov 03, 2020 10:46 am
Has thanked: 1 time
Been thanked: 2 times

Re: 4K Upscaled Persona 5 Royal Bustups Mod

Post by shst » Wed Dec 09, 2020 7:54 am

LagBeam wrote:
Wed Dec 09, 2020 6:31 am
Have you used it with TTA on or off? if off, then it should be indistinguishable.
It wasn't enabled by default, so no. I'veenabled it, and processing an image took about an hour instead of regular 8 minutes. Results are similar to yours. I didn't change tilesize since I'm not aware what exact values I need.

What I'm much more concerned about are outlines. I'm searching for an automatable way to preserve transparency in a separate file, since if you strip transparency from image, processing it with a white background, everything looks fine (example, only used the first model to not waste more time).

EDIT: once again, Imagemagick saves the day, serving as a workaround. The procedure is as follows:
  1. Create an image with stripped transparency, and a separate image with alpha layer. If done right, you'll have two files: one with character sprite, and one with white silhouette, both of which have black background.

    Code: Select all

    convert b008_002_04.png -alpha off noalpha.png
    convert b008_002_04.png -alpha extract alphamask.png
    
  2. Upscale both of those images with whatever tools you use, in my case realsr-ncnn-vulkan. You don't need to use deblur or any other models on a silhouette image, single upscale would be enough.
  3. After upscale is done, combine the opaque image with an alpha layer.

    Code: Select all

    convert noalpha.png \( alphamask.png -alpha copy -channel rgb -evaluate set 100% +channel \) -channel rgba -alpha on -compose multiply -composite result.png
    
Result is something like that: https://de.catbox.moe/llv89j.png. Once again, I've only used the 4x model to not wait too much before explaining the procedure.

LagBeam
Posts: 17
Joined: Tue Aug 04, 2020 6:03 am
Location: Russia
Has thanked: 5 times
Been thanked: 1 time

Re: 4K Upscaled Persona 5 Royal Bustups Mod

Post by LagBeam » Wed Dec 09, 2020 10:02 am

I want to post my quick thoughts on alpha-channeling work you've done. What are you aiming for by stripping transparency? ability to use software that doesn't process it right?
In this case I guess that's fine. If goal was to reduce outlines to a "sharp" shift in transparency disregarding how it will be processed on the end device (PS3/RPCS3) - I guess that's fine, but when you will be making final bustups you can use DXT5 compression for each and every resulting image (unlike what devs did - DXT5 for big one and DXT1 for small ones). And finally if you want to make outline smaller - I checked the image and it simply isn't smaller - it's bigger.

About performance - tile size should be investigated based on your VRAM - just increase value until running into errors then back down and run as is (but don't create other VRAM issues meanwhile).
Your performance is unfortunately tanked by your RX460 - in which case I'm fully willing to leverage upscaling job by utilizing my RX5700XT.

Post Reply