WWDC24: Create custom visual effects with SwiftUI | Apple - バイリンガル字幕
Hello, and welcome to Create Custom Visual Effects in SwiftUI.
My name is Philip, and I'll be joined in a bit by Rob.
Together, we're going to share how you can create visual effects to make apps that are more expressive and pleasant to use.
Building great app experience is often the result of making many small improvements, small details that in aggregate make a big difference.
Visual effects can play an outsized role in how an app is used and perceived.
They can show that a feature is working as expected.
Add personality to your app's surface.
And focus attention on something important that's happening.
When I'm creating a new visual effect, I'm often not sure what will work until I can.
I need to experiment, tinker, and play with ideas until things feel right.
In this session,
Rob and I are going to build out a number of examples and explore how using SwiftUI,
you can create custom scroll effects,
bring rich color treatment to your apps with Compose custom view transitions,
create beautiful text transitions using text renderers, and write metal shaders to create advanced graphic effects.
We're going to start with something we're all probably familiar with, scrolling.
So much of our app experiences are collections of items that we scroll through.
whether they be photos, videos, or text blocks.
ScrollViews are everywhere.
Here, I have a simple collection of photos inside a horizontal scroll view.
In SwiftUI, scrollViews provide a lot of automatic support for common use cases.
Here, I'm using a paging behavior to get a pagination effect.
This is fine for a standard scroll view, but I'd like to create something a bit more unique.
Let's take a look at a single photo.
SwiftUI's scroll transition modifier can be used to change a standard collection of elements into something custom.
Scroll exposes the content I'd like to transition,
as well as a I can use these values to change the rotation and offset of each photo in my scroll
view based on its position.
As I scroll, the photos on the leading and trailing edges are rotated, creating a circular carousel effect.
I can use the value property to determine how far off screen my images and use that for rotation.
And if my view is fully on screen, the isIdentity property will be true.
This rotation effect is nice, but it doesn't really fit the metaphor I'd like to use here.
I'd like each of these cards to feel as if they're a window that you can look through.
By changing the modifiers, my scroll transition is updating.
I have the ability to completely change the feel of the scroll view to create a parallax effect.
Here, I'm using the scroll transition to change the X offset of the image, but not the shape that's clipping it.
ScrollTransition can be used to manipulate this content in tons of different ways.
I can take this modifier and put it on any content that I want to update based on the scroll value.
Here, I've added a scrollTransition to a text caption underneath my image so that it fades out and offsets to amplify the momentum of the scroll view.
Scroll transitions are a great way to build interesting, unique scroll experiences.
Sometimes, though, you need a bit more control over how a view's position or size affects its visual appearance.
Here, I have a simple collection of grocery items that I can scroll through.
Right now, each item is the same color, which looks monotonous.
I can add a visual effects modifier, which provides access to a content placeholder and a proxy.
The content placeholder works the same as in the scroll transition.
The proxy can give me the geometry values of the view.
I can take the location of the view from the proxy and use it to change the hue of my view,
which creates a nice gradient effect.
The lower the view on my device, the stronger the hue rotation.
The visual effects modifier lets you change visual properties based on the view position and size in a performant way,
which means it's great for use in scroll views.
Instead of changing the color, I could change other visual properties.
Here, I'm taking the same Y position of my shape, and using it to offset your
fade and blur an element as it gets to the top of the scroll view.
The scroll transition and visual effect modifiers are great ways to create custom scroll view effects.
You can use them to create scroll views that adjust scale based on the position of an element on the screen.
You could use them to change the perspective by using different transforms like rotation and skew.
Use offsets to create stacking behavior or adjust color properties like brightness, saturation and hue to create emphasis and provide clarity.
It's not always clear, though, if an effect is right for your app or if it's distracted.
It's helpful to spend time living with visual experiments.
Visual effects should be pleasant to use well after the novelty has worn off.
Testing your effects over time and in different contexts will help reinforce if an effect is working or where it still needs improvement.
Next, let's talk about how you can bring color effects to your app.
Color place an important role in an interface.
It can help give your app an identity, focus attention, or clarify intent.
SwiftUI has a lot of tools for bringing color into your app.
Their support for different gradient types, color controls, blend and much more.
New in SwiftUI is support for mesh gradients.
Mesh gradients are useful when you want a dynamic background or need to add some visual distinction to a surface.
Mesh gradients are made from a grid of points.
Each of these points has a color associated with it.
SwiftUI interpolates between these colors on the grid to create a color fill.
These points can be moved to create beautiful color effects.
The colors blend together smoothly, and points that are closer together have a sharper color transition.
In order to create a mesh gradient, I'll use the new mesh gradient view.
I'll define the rows and columns of my grid using the width and height parameters.
In this case, I'll use a 3x3 grid.
Next, we're going to define where the x and y coordinates on this 3x3 grid are located
Points in the grid are defined using SimD2 float values.
When used as a view, these floats take a value from 0 to 1 on the X and Y axes.
Finally, I'll add a corresponding color for each of these points.
This creates our mesh gradient.
Right now, it looks a bit like a linear gradient.
If I move the X and Y coordinates of the center point, the colors move to match the new position.
Mesh gradients are a nice way to add color effects to your app, and you can use them to create all sorts of visual effects.
But you could also use them to match a with imagery, or even signal that something has changed through a mesh gradient animation.
Play around with values like the position of control points,
grid size,
and control Tweaking parameters and exploring the edges of what's visually possible will lead you far beyond any ideas you have at the beginning.
So be bold, turn the dial up to 100, and make something new.
Next, let's talk about creating custom transitions.
Our interfaces are a portal into what our app is doing behind the scenes.
And transitions are a way to communicate the changes that are happening.
Transitions are useful when you want to show new views, or remove views that aren't needed anymore.
They can help provide context as to what changed and why the change was changed.
Sometimes, these transitions are due to the tap of a button or the drag of an element.
Sometimes, they are triggered by the behavior of someone else using an app.
I have an avatar view that shows and hides based on that person's online status.
If they're online, I'd like their avatar to show, and otherwise it should be hidden.
Right now, it just appears and disappears.
This is a bit jarring, so let's add a transition.
We can apply one of SwiftUI standard transitions, like scale.
To have it scale up, and down as it enters and exits.
If we want to chain multiple transitions, we can use the combined method to add another.
Let's combine our scale transition with opacity.
This is looking better, but what if we want something more custom?
In order to create a custom transition, I'll create a new struct.
I'll call it twirl.
This will conform to the transition protocol.
The transition body function takes a content and phase parameter.
The content parameter works the same as what I shared for the scroll views.
as a placeholder for the content I want to transition.
I can use the Phase value to check if a view is currently being shown and use that to conditionally style my view.
For scale, I'd like it to be at full scale when it's shown, and at half the scale when it's not.
For opacity, I'd like my element to toggle between fully visible and hidden.
I can attach my custom transition to my view and check out the results.
Back in my custom transition, I'd like to add blur so that it looks like the avatar is coming in and out of focus.
and also add some rotation so that it spins.
I can check my phase value for whether or not a view will appear or did disappear.
This will let me continue rotating the same direction on exit by using a negative value.
I'll add a brightness modifier so that when the view is entering, it has a bit of shine and catches attention.
With a few small adjustments, we're able to make our interface element respond to changes in a graceful way.
Transitions can be used in many types of scenarios,
to ease an element into view as it's loading, introduce important piece of information, or make a graphical element feel dynamic.
A good transition will fit naturally within its larger context and not feel like it was tacked on.
Looking at your app holistically can help you decide what transitions are the right fit for your app.
Speaking of transitions, I'll hand it off to Rob to talk about text transitions.
Thanks Phillip, let's dive in.
Philip already explained how to use the built-in SwiftUI transitions to animate in views, like this opacity transition.
While I could definitely spice it up using the built-in modifiers, I like to try animating a text in line by line.
To do this, I'll use text renderer, a new API introduced in iOS 18 and aligned releases.
Textrendra is a powerful new protocol that allows you to customize how SwiftUI text is drawn for an entire view tree.
This enables a whole new range of custom text drawing possibilities, but the one I'm most excited about is animation.
The core of the Textrendra protocol is the draw layout in method.
Its arguments are text.layout and a graphics contest.
Text.layout is what allows us to access the individual components of the text, its lines, runs, and glyphs.
The graphics context is the same type that is used by the canvas view.
Check out Adrich Graphics to your SwiftUI app if you'd like to know more about how to draw with it.
For a minimal text renderer,
I only need to iterate over the individual lines of the layout using a for loop and draw them into the context.
This will give me the default rendering behavior.
To drive my transition, I add three properties to my text renderer.
A lapse time,
how much time has passed so far, element duration, how much time should be spent animating an individual line of character and total depth?
iteration.
How much time the entire transition will take.
To have SwiftUI automatically animate the elapsed time value for me, I implement the animatable protocol.
It's simple to adopt in this case by forwarding the animatable data property to elapsed time.
Now, I can start iterating on my animation.
First, I'll try animating a line by To distribute the available time evenly across the animation,
I need to calculate the amount of delay between two consecutive lines using this helper function I called element delay count.
Next, I enumerate all lines and calculate the relative start time based on their index and that delay value.
The time that has passed for an individual line is the overall elapsed time minus the element's individual time offset.
I also clamped this value.
Next, I create a copy of the current graphics context.
This will make sure that individual calls to my helper function will not affect each other since graphics context has value semantic.
Finally, I call my helper function to draw the individual line.
This is where the magic happens.
Before I draw the line, I update the properties of the graphics context that I want to animate.
To make this easier, I also calculate a fraction of progress value.
First, I want the line to fade in, so I calculate the quick opacity ramp.
At the same time, I decrease its blur radius to 0 to give the impression that the line manifests from a diffuse state.
The initial blur radius is based on the height of the line to direct from the line's typographic bounds property.
Lastly, I animate a on the Y axis using a spring.
I started a Y position that has shifted upwards based on the length of the line's descender.
Finally, I draw the line using the new draw options method.
By opting out of sub-pixel quantization,
I can avoid jitter as my spring In order to use the renderer to animate in text,
I implement a custom transition like Philip explained earlier.
By experimenting, I found that 0.9 seconds feels like a good duration for my use case.
However, I need to consider that it could already be an animation on the current transaction.
For example,
when this transition was triggered from a call to with animation, Using the transaction body view modifier, I can overwrite the animation when appropriate.
This way, I can ensure an even, linear pacing for every line.
Then, I used new text render view modifier to set my custom renderer on the view being transitioned in or out.
Here's the transition and action.
I like it.
it, but I don't love it.
It's dependent on the number of lines which can change based on locale or dynamic type size.
Also, it doesn't quite capture my excitement for visual effects.
Let's try animating every cliff individually.
To do that, I need to iterate over the text.layout They represent the smallest unit of layout, like glyphs or embedded images.
A text.layout is a collection of lines, a line is a collection of runs, and a run is a collection of run slices.
Therefore, using this helper method called flat and run slices, I need to iterate
over the run slices instead and get to keep almost all of my logic.
I also need to revisit my helper function,
but all I need to do here is to change the type and name of its line argument to a run slice.
Here's the result.
I think that's better, but now I have the opposite problem.
There's very little time left in the animation to dedicate to an individual glyph.
This reduces the overall impact, making the transition feel less fun and a little samey.
I think I need to dial it back a little.
Instead of animating everything the same way, I'll focus only in the words visual effects.
That way, I can use the transition not only to bring in the content, but also to emphasize what's important.
To do that, I'm using the new text attribute protocol, introduced alongside text renderer, in iOS 18 and Alignable.
By implementing this protocol, I can pass data from my text to my text renderer.
Applying the attribute is very simple.
Using custom attribute text modifier, I mark the words visual effects using my custom emphasis attribute.
Because it's only used to mark a range of text, I don't actually need to add any member variables to my text attribute struct.
We're visiting the drama if at one last time I now iterate over the flattened runs of my layout.
I check the presence of the emphasis attribute on the run using a subscript with the attribute type as its key.
If the attribute is present, iterate over the slices in the exact same manner as I did before.
Or, if the attribute is absent, I'll quickly fade in the run over the course of 0.2 seconds.
Here's the final result.
This is much better.
The transition now really emphasizes visual effects.
Text renderer opens a whole range.
By breaking a view into smaller components that animate individually, you can build more expressive animations and visual effects.
And there's another powerful graphics API in SwiftUI that offers even more fine-grained control.
Shaders.
Shaders are small programs that calculate various rendering effects directly on your devices.
SwiftUI uses shaders internally to implement many of the visual effects Philips showed you earlier, like your new mesh gradients.
With SwiftUI shaders introduced in iOS 17 in the line releases,
you'll be able to unlock the same level of performance and write your own impressive effects.
You instantiate a shader in SwiftUI by calling a function with its name on Here,
you can also pass additional parameters to your shader function, like colors, numbers, or an image.
When you apply this effect to a view,
using the layer effect view modifier, SwiftUI will call your shader function for every single pixel of your view.
That's a lot of pixels.
To make this possible in real time, Shader's run on your device's GPU, which is optimized for highly parallel tasks such as this.
However, because of the specialized nature of GPU programming, the Shader's themselves cannot be written in Swift.
Instead, they are written in the metal shading language, or metal for short.
Here's the corresponding metal file for the Shader I showed you earlier.
The name of the shader function matches the invocation on shader library.
This is the function that SwiftUI will execute on the GPU for each of your views pixels.
And when it does, the position argument refers to that pixels location.
The layer argument, meanwhile, is a representation of your views content.
You can sample the layer to obtain its contents,
but you must stay within the max sample offset that the shader was instantiated with relative to position.
SwiftUI also resolves and converts types, such as color, to representations that can be used in metal.
Here, my pink color gets converted to a half-four.
Metal makes heavy use of value.
A half-four is a four-component vector of 16-bit floating point numbers.
This type encodes the red, green, blue, and alpha components of the color.
Similarly, float two is a two-component vector of 32-bit floating point numbers and frequently used for 2D points or dimensions.
In SwiftUI, shaders can be used for custom fills and three kinds of effects, color effects, distortion effects, and layer effects.
Of the three effects,
layer effects are the most powerful and effectively is super set up the other two, so I'll show you how to write a letter.
I think it's going to be a great way to add some character to my app.
Currently, I have this push effect installed on my view that is triggered whenever I tap it.
View down using a spring, then immediately pops back up.
This gives me direct feedback from interaction, but the animation does not respond to where I touch it.
This makes it feel lifeless and stiff.
Instead, I would like it to look more like this.
Whenever I touch the view, the scale effect spreads outward from the touch location, affecting every pixel of my view differently.
With SwiftUI Shaders, I now have the tools I need to make an effect like this reality.
To implement this effect, I add a new shader function to my metal file that I call ripple.
I add the two arguments required by the layer effect API, position, and layer.
I've already worked out the formula that describes each pixel's output.
It's a function of the point in which the view was touched, how much time has passed, as well as these four parameters.
calculated distortion for this pixel leaving me with this new position value.
This is where I sampled a view.
After some tweaking based on the strength of the distortion, I returned the modified color.
Next, I need to call this shader function from SwiftUI.
To do that, I created this view modifier called ripple modifier that exposes all the parameters of the shader function to SwiftUI.
In its body content method, it instantiates the shader and applies it to its content.
Because shaders have no concept of time, we also need to drive the animation from SwiftUI.
Here's how I do this.
that.
I've heard a second view modifier called ripple effect.
The keyframe animator view modifier makes it easy to run animations based on external changes, like gestures.
I animate the elapsed time from zero to its final duration value whenever the trigger of value.
This way,
at every step of the animation, ripple modifier will be passed the current time and the origin point at which I touch the view.
But wait,
I never assigned values to the four parameters I showed you earlier,
and I'll be honest, I have no idea what values would look good here.
I'll just have to experiment, so I built myself this debug UI.
Because Ripple modifier does not perform any animation itself I can use it to scrub forwards and backwards for the animation interactively.
This way I can dial in the right parameters for my shader function on my phone or inside the next code preview.
Building great experiences requires a lot of trial and error and debug UI is a great way to iterate on complex animations.
This can mean exposing parameters or drawing an overlay that visualizes intermediate values.
Getting immediate feedback like this is incredibly powerful and makes it easier to quickly iterate.
That's important because there are so many possibilities for what you can create with shaders.
You can use shaders to create an animated fill to a texture to your app.
You can combine shaders and text renderer to apply distortion to text or use them to create gradient maps for your unique photo effects.
In this video,
we looked at a number of ways to create visual effects with SwiftUI and we encouraged you to put your own spin on these ideas.
Experiment with custom scroll effects to set your app apart.
Add a splash of color with mesh gradients.
Treat app to some custom V transitions.
Make text, come alive with the new text render API, and build a wild new experience with a metal shader.
Use these tools to invent something new.
Thank you for watching.
さらなる機能をアンロック
Trancy拡張機能をインストールすると、AI字幕、AI単語定義、AI文法分析、AIスピーチなど、さらなる機能をアンロックできます。
主要なビデオプラットフォームに対応
TrancyはYouTube、Netflix、Udemy、Disney+、TED、edX、Kehan、Courseraなどのプラットフォームにバイリンガル字幕を提供するだけでなく、一般のウェブページでのAIワード/フレーズ翻訳、全文翻訳などの機能も提供します。
全プラットフォームのブラウザに対応
TrancyはiOS Safariブラウザ拡張機能を含む、全プラットフォームで使用できます。
複数の視聴モード
シアターモード、リーディングモード、ミックスモードなど、複数の視聴モードをサポートし、バイリンガル体験を提供します。
複数の練習モード
文のリスニング、スピーキングテスト、選択肢補完、書き取りなど、複数の練習方法をサポートします。
AIビデオサマリー
OpenAIを使用してビデオを要約し、キーポイントを把握します。
AI字幕
たった3〜5分でYouTubeのAI字幕を生成し、正確かつ迅速に提供します。
AI単語定義
字幕内の単語をタップするだけで定義を検索し、AIによる定義を利用できます。
AI文法分析
文を文法的に分析し、文の意味を迅速に理解し、難しい文法をマスターします。
その他のウェブ機能
Trancyはビデオのバイリンガル字幕だけでなく、ウェブページの単語翻訳や全文翻訳などの機能も提供します。