Page 1 of 1

AA coloring issues

Posted: 24 Nov 2020, 18:57
by Peter Wassink
Precise coloring with Anti-Alias can be problematic
with the paintbucket set to anti alias we get a 'balooning'effect on the fill

this also happens when exporting the C&T layer set to anti-alias


This Anti-Aliasing effect seems to me to be a bug, or a symptom of some non optimized algorythm.
It was reported earlier i believe, cant relocate that but ran into this issue again.

The fill tool with AA or the Col&Texture layer set to extract with AA, both give the same distortion of the filled shape,
making it look as if its blown up.

i would think that a perfect square should look the same with or without AA,
only when a line has an angle other then 0° 90° 270° or 360° should AA kick in?
to me it seems weird that the AA would cause a square to blow up like a balloon even growing outside of its fill boundaries.





Another thing i noticed with is that the fill border goes up to the border on top and Left of the fill.
But it goes over 1 pixel to right and bottom boundary?
Why is the fill asymmetrical?
You can see it in this image, where i put the layer with black boundaries at 30% opacity

colo&texture-AA-issue.png
colo&texture-AA-issue.png (16.04 KiB) Viewed 14876 times

Re: AA coloring issues

Posted: 25 Nov 2020, 09:14
by Xavier
Yes, i think you reported it in the beta thread. Thanks anyway, it's better here :)

Re: AA coloring issues

Posted: 25 Nov 2020, 09:36
by Peter Wassink
yes we are currently struggling with this issue.
And when i revisited my post, i saw you suggested i put it here.

i mean this a-symmetrical filling.
we use the C&T and these edges cause us problems in compositing.
colo&texture-AA-issue2 .png
colo&texture-AA-issue2 .png (12.49 KiB) Viewed 14845 times

Re: AA coloring issues

Posted: 25 Nov 2020, 09:39
by Xavier
The multiply blending mode is a pain to do. I also have an aliasing issue...

Re: AA coloring issues

Posted: 25 Nov 2020, 09:49
by Peter Wassink
Xavier wrote: 25 Nov 2020, 09:39 The multiply blending mode is a pain to do. I also have an aliasing issue...
Yes i imagine Aliasing is hard to get right
i'm not sure what multiply has to do with this issue specificly, could you elaborate?

Re: AA coloring issues

Posted: 25 Nov 2020, 10:10
by Xavier
Peter Wassink wrote: 25 Nov 2020, 09:49 i'm not sure what multiply has to do with this issue specificly, could you elaborate?
Completely unrelated, just to let you know on which topic I am currently :)

Re: AA coloring issues

Posted: 25 Nov 2020, 10:11
by Xavier
Peter Wassink wrote: 25 Nov 2020, 09:36 yes we are currently struggling with this issue.
And when i revisited my post, i saw you suggested i put it here.

i mean this a-symmetrical filling.
we use the C&T and these edges cause us problems in compositing.
colo&texture-AA-issue2 .png
Thanks for the detail level, no worries I'll have a look.

Re: AA coloring issues

Posted: 21 Dec 2020, 22:19
by artfx
I wonder if this is why I notice in videos about Japanese studios (that use TVPaint) that they use no antialiasing, on their lines or colors... none... nowhere.

Back in the day when we had a 2D team at the studio in Shanghai, I don't know what software they were using for ink and paint, but I was surprised to see the same thing. No antialiasing on anything.

I wonder what resolution they must work at to make that look good.

Re: AA coloring issues

Posted: 24 Dec 2020, 09:37
by Hironori Takagi
artfx wrote: 21 Dec 2020, 22:19 I wonder if this is why I notice in videos about Japanese studios (that use TVPaint) that they use no antialiasing, on their lines or colors... none... nowhere.

Back in the day when we had a 2D team at the studio in Shanghai, I don't know what software they were using for ink and paint, but I was surprised to see the same thing. No antialiasing on anything.

I wonder what resolution they must work at to make that look good.
There are two reasons why we don't color with antialiasing.
(1) It takes time to color.
(2) When compositing, use the color area as mask.
After adding textures and effects with each color area, antialiasing is handled with plug-in effects.

Re: AA coloring issues

Posted: 28 Dec 2020, 11:26
by artfx
Hironori Takagi wrote: 24 Dec 2020, 09:37
artfx wrote: 21 Dec 2020, 22:19 I wonder if this is why I notice in videos about Japanese studios (that use TVPaint) that they use no antialiasing, on their lines or colors... none... nowhere.

Back in the day when we had a 2D team at the studio in Shanghai, I don't know what software they were using for ink and paint, but I was surprised to see the same thing. No antialiasing on anything.

I wonder what resolution they must work at to make that look good.
There are two reasons why we don't color with antialiasing.
(1) It takes time to color.
(2) When compositing, use the color area as mask.
After adding textures and effects with each color area, antialiasing is handled with plug-in effects.
Wow! Thanks for answering and explaining that. I didn't know that antialiasing could be added later. This is very good information.

Re: AA coloring issues

Posted: 28 Dec 2020, 20:31
by D.T. Nethery
Hironori Takagi wrote: 24 Dec 2020, 09:37 There are two reasons why we don't color with antialiasing.
(1) It takes time to color.

(2) When compositing, use the color area as mask.

After adding textures and effects with each color area, antialiasing is handled with plug-in effects.
.

Are the plug-ins to add antialiasing to the image plug-ins for TVPaint or in another application ?

Re: AA coloring issues

Posted: 29 Dec 2020, 10:37
by Hironori Takagi
D.T. Nethery wrote: 28 Dec 2020, 20:31
Hironori Takagi wrote: 24 Dec 2020, 09:37 There are two reasons why we don't color with antialiasing.
(1) It takes time to color.

(2) When compositing, use the color area as mask.

After adding textures and effects with each color area, antialiasing is handled with plug-in effects.
.

Are the plug-ins to add antialiasing to the image plug-ins for TVPaint or in another application ?
For example, we use After Effects plug-in "OLM Smoother".
https://olm.co.jp/rd/technology/tools/?lang=en

Re: AA coloring issues

Posted: 29 Dec 2020, 15:19
by D.T. Nethery
Hironori Takagi wrote: 29 Dec 2020, 10:37
D.T. Nethery wrote: 28 Dec 2020, 20:31
Are the plug-ins to add antialiasing to the image plug-ins for TVPaint or in another application ?
For example, we use After Effects plug-in "OLM Smoother".
https://olm.co.jp/rd/technology/tools/?lang=en

Thank you. I've downloaded it and will try it.

Do you apply the OLM Smoother to a single composite layer (characters and backgrounds) in After Effects or is OLM Smoother applied to
separate layers of characters and backgrounds in AE before doing the final export to .MOV or .MP4 from Adobe Media Encoder ?