Lyrics 3.0 - Abusing CSS Custom Properties to create themeable synchronised lyrics

This post is part of a 3 part series. Consider reading part 1 and part 2 first, although this post should still make sense on its own.

Some people like to see the finished product first. If this is you, you can see this project in action at lyrsync.yoshiwalsh.me. If you prefer the suspense, read on, I’ve put another link near the end of the post.

Inception

So my goals were:

1. Have one project that can support many songs.
2. Allow each song to have fully customised formatting.

Architecture changes for supporting multiple songs

Let’s get this out of the way first, since it’s not particularly interesting. Now that the project was going to be more than a simple once-off toy, I needed to give the project some proper structure.

Each song would have its own page, and those pages would be very similar. I didn’t want to have to copy/paste a bunch of HTML for each page, so I needed a templating engine. On top of that, code cleanliness needed to be more of a priority. (Especially CSS, for reasons that will become apparent later.)

So it was time to bring in my friends Handlebars, TypeScript and Sass. I’ve wasted enough time trying to configure webpack (and lamenting the fact that it doesn’t support HTML entry points) that I never want to touch it ever again, so instead I reached for Parcel.js. Parcel.js isn’t perfect, but for simple projects like this it’s dead-simple to set up.

I created a Handlebars partial for the playback page and then used that partial to make a page for each song. (Initially just étoile et toi.) I also ported the existing JavaScript to TypeScript and the CSS to SCSS.

Theming system

When I faced the challenge of allowing each song to have its own theme, the obvious answer was to give each song its own stylesheet. This would be possible, since (unlike with LyrTube) the lyrics were already displayed using DOM elements.

For some aspects (font, colour, positioning) this was enough. The difficulty was the animations.

CSS3 introduced some powerful tools for designing animations. Unfortunately, JavaScript’s control over these systems is very limited. You can start/stop animations & transitions by adding or removing classes, but that’s about the extent of it. You can’t control the speed of animations, you can’t pause animations, you can’t seek within an animation. This causes some issues with trying to sync them to music.

For example, if I’d used CSS transitions and the user seeked to a point later in the song, a bunch of transitions would’ve played to go between those two points, which would’ve been fairly confusing. Instead, I just wanted to be able to jump right to that point without the intervening animations playing.

But if I’m completely honest, this is just a rationalisation for why I didn’t want to use CSS transitions or animations. The real reason is that it felt wrong. CSS transitions/animations rely on JavaScript setting them off at the right time and then they play by themselves. In order to make this work with music, when I detected an update to the currentTime of the video I would have to compare it to the previously seen currentTime value and start any animations that were scheduled for between those times. In contrast, the way that LyrTube and étoile et toi worked was that they looked at the currentTime of the video and moved everything to be at the exact places that they should’ve been at at that time. I’m sure there’s an elegant technical term to describe these two approaches, but I’m sadly not familiar with it. Perhaps we can call the first approach ‘event-driven’ and the second approach ‘parametric’. If you know more appropriate terms, please contact me and let me know!

Anyway, the second (‘parametric’) approach seems much more correct to me, so I was determined to keep using it. This meant CSS transitions & animations were out.

The way animations worked in étoile et toi was that they were wholly driven by JavaScript. JavaScript updated all the positions, opacities and transformations. But I was also set on not needing any per-song JavaScript, so this option was out as well.

When there’s no feature that officially does what you need it to, it’s time to start abusing other features until you get what you want.

Abusing CSS

Introduction to calc()

CSS has a function called calc() which can perform simple arithmetic. The purpose of this is to allow you to do things like positioning an element 20px below the middle of the page. top: calc(50% + 20px); As someone who’s been writing CSS since long before calc, I am incredibly grateful for this ability, as in the past simple tasks like this often required multiple extra elements to achieve.

Introduction to Custom Properties

CSS also has a fairly new feature called Custom Properties. CSS has a large number of standard properties (such as display, font-family, width, etc…), but with Custom Properties you can make your own properties, they just have to be prefixed with --.

I remember when I first heard about this feature I immediately thought how useful it would be for allowing customisation of psuedo-class styles like hover. If you have 3 links on a page and you want each to be a certain colour, it can easily be accomplished via inline styles:

<a style="color: red">test</a>
<a style="color: green">test</a>
<a style="color: blue">test</a>

But now let’s say you also want to customise the hover colour of the link. Not so easy:

<a style="color: red" class="link1">test</a>
<a style="color: green" class="link2">test</a>
<a style="color: blue" class="link3">test</a>
<style type="text/css">
    a.link1:hover { color: RebeccaPurple; }
    a.link2:hover { color: BlanchedAlmond; }
    a.link3:hover { color: LightGoldenRodYellow; }
</style>

This is fine if you’re a web developer (you should really be using CSS for styling your links anyway) but what if you’re building a system that allows users to choose the colour for their links? It’s easy to insert a inline style on each link, but it’s a bit annoying to have to give every link a unique class/id and then generate CSS to apply their hover styles. This is something I’ve done before, and while it’s not difficult, it was irritating that it was necessary. Well, thanks to custom properties it’s no longer necessary:

<style type="text/css">
    a:hover { color: var(--hover-color); }
</style>
<a style="color: red; --hover-color: RebeccaPurple">test</a>
<a style="color: green; --hover-color: BlanchedAlmond">test</a>
<a style="color: blue; --hover-color: LightGoldenRodYellow">test</a>

Putting them together

JavaScript can be used to set inline styles on an element, including setting values for custom properties. So I got to thinking, what if I made a custom property for --current-time and used JavaScript to update it once per frame with the current time in the song. Then I could write CSS that takes the --current-time value and uses calc() to calculate the position/style of each element. This way, CSS has full control over the animation.

I built a proof-of-concept to test whether this would work in practice. I was particularly concerned about the performance of it. I think that neither calc() nor Custom Properties were designed to be updated 60 times per second. The proof-of-concept simply animated the opacity of each card and word using this technique. In a fairly short amount of time I had this working, and so I felt pretty confident about this approach.

My next goal was to fully port étoile et toi to the CSS driven approach, making the JavaScript fully song-agnostic.

It all comes tumbling down

I ran into issues when I tried to animate the position of the words. In étoile et toi, the words gently slide up/down from the centre line while they fade in. It was while trying to replicate this slide effect that I ran into a fatal flaw with my approach.

You see, with opacity the valid values are from 0 to 1. If you specify a value lower than 0, it’s treated as 0. If you specify a value higher than 1, it’s treated as 1. So I could just do opacity: calc((var(--current-time) - var(--card-time)) / var(--card-fade-duration)).

But if I used this simple approach for the position animation, I wouldn’t be able to stop the text from moving once it reached its destination. It would just keep sliding forever.

Now you might be thinking that this was obviously going to be a problem and that I should’ve thought of it from the start. And I had, I’d planned for exactly this. I just needed to use a clamping function, and CSS has one.

But when I went to use clamp(), it didn’t work. Firefox Developer Tools reported “invalid property value”, and the position transformation was not applied.

Not only is clamp() not supported in any browser, it’s part of CSS Values and Units Module Level 4, which is still in Editor’s Draft status. So it’s not likely to be supported in browsers for a while.

At this point, swear words were uttered. By this point I’d spent about 6 hours on this, and suddenly it seemed it was impossible.

If clamp() was supported in just one browser I could’ve proceeded with the project and just said it relied on cutting edge features and only worked in that browser. But because it wouldn’t work in any browser, there was no way for me to even test it, let alone show it to other people.

The greed of a pig

Desperation struck. I just needed to build a clamp function! CSS doesn’t have any way to define custom functions, but if I could build a calc() expression for clamp then I could use Sass to let me re-use it. How hard can it be to implement a clamp function using simple arithmetic?

Well calc() only supports addition, subtraction, multiplication, and division. Not only is it difficult to build a clamp function with these, it appears to be provably impossible.

But it’s possible to build a half-decent approximation that works for a certain range, as long as you’re willing to use a very long expression. Maybe that’s good enough?

I started working on this, but I stopped before I got anything working because I realised I had another concern with this approach. Performance.

Because the best I could achieve with this method was an approximation, it wouldn’t be stable. So instead of outputting 1 if the value was higher than 1, the clamp function might output 1.0001, 1.0002, 0.9999, etc… These minute differences would be detected as changes in the CSS and would cause the element to be repainted.

Obviously the words that are currently animating will need to be repainted, but with an unstable clamp function every single word in the whole song needs to be repainted 60 times per second, even if they aren’t moving. Maybe this would’ve been fine, but I think this probably would’ve caused issues on some browsers/platforms.

So I gave up. I stashed my changes and abandoned it. I vowed to return to the project once CSS clamp() was implemented in browsers.

And that’s where the story ends. Well, that’s where the story ended. Until one night, while I was trying to sleep, and out of nowhere I had it. I knew how to make it work.

Fresh hope

I spent the entire following day working on a new proof-of-concept. It’s impossible to make a clamp function in CSS, but did I have to? The reason I wanted to do everything in pure CSS was so that the animations could be fully customised with CSS. But what if the CSS could influence the behaviour of the JavaScript? CSS and JS working together to achieve the result. The behaviour of the JavaScript would be fully customisable using CSS.

So I designed a system that I called Timers. It works like this:

1. The CSS defines a custom property requesting certain timers.
2. The JS reads this value using getComputedStyle() once, just after the lyrics have loaded. getComputedStyle() is an expensive function, so I can’t afford to use it every frame.
3. The JS calculates a value for each timer every frame and stores that value in another custom property.
4. The CSS can then use that custom property in calc expressions.

For example:

.card {
    --card-timers: --fade-in start -0.5 start 0 linear none none;

    opacity: var(--fade-in);
}

This defines a timer that’s based on the start time of the card. It increases linearly from 0 to 1, starting half a second before the start of the card and ending at the start time of the card. The value from the timer is put into the --fade-in custom property and is then used to control the opacity.

By the end of the day, I had rewritten étoile et toi to use this new system, and it worked. I’m probably now on a list somewhere for cruelty-to-browsers, but it was worth it. I excitedly showed a few friends, and they said “you’ve already shown me this”. Ha, but that was the point! The result looked exactly the same, but the inner workings were now much more flexible.

Incremental improvements

After that, the next step was to add a second song. Finally, time to sync Puppe. An issue with Puppe is that it’s not on YouTube anywhere. But it is on Vimeo. So I needed to add Vimeo support.

A while ago I read about Popcorn.js, a project Mozilla had to provide a unified API for embedding media, whether it was from YouTube, Vimeo, SoundCloud, or somewhere else. So I replaced my usage of the YouTube API with Popcorn.js. Popcorn.js is no longer maintained, so I found it a little confusing to get running, but I managed.

Another benefit of Popcorn.js is that its currentTime() function works as you would expect, they smooth over the weird jittery behaviour that I mentioned in part 1. Because of this, I was able to remove my custom timekeeping code and just use currentTime().

I added Komm, süßer Tod next, fixing a couple of bugs at the same time. This song starts with a fairly long instrumental, during which a black screen was displayed. I didn’t want people to think that it wasn’t working, so I added progress bars to long instrumentals sections. I was able to add these progress bars using only CSS, I didn’t need to write any new JavaScript, so this proved the flexibility of the animation system. In Komm, süßer Tod I also demonstrated the capabilities of the system by applying a 3d tumbling effect to some of the lyrics.

I decided to add a system for adding arbitrary timed events into lyrics. I put this to use with Rammstein’s Du Hast, using this new cues system to add flashing horizontal bars during the keyboard riff.

Perhaps the most significant improvement that I made was that I finally wrote my own transport controls. Now it was easy to play/pause, seek, mute/unmute, and fullscreen the presentation.

Conclusion

You can find the finished product here. I’ve fully open-sourced it, so you can also find it on GitHub here.

At the end of this project I am somewhat left wondering what I’ve achieved here. A lyrics engine that requires authors to write themes in CSS. A friend asked me if I was going to add a GUI to allow people to design themes, and I realised that due to the design decisions I made this would be very difficult. In order to make a GUI that exposed the full power of the animation system, I’d need to build something similar to Unity’s Shader Graph, an endeavour I’m certainly not planning on undertaking. Someone on Reddit pointed out that the same results could’ve been achieved with ASS subtitles, which can be authored with a GUI in Aegisub. (Of course, ASS subtitles aren’t natively supported in browsers, more on that in a future blog post.)

So what is it that I’ve created? Maybe a programming horror story, an abomination which I should be ashamed to have published? Maybe it’s a kind of esoteric joke, similar to The Simpsons in CSS?

If I absolutely had to come up with a reason why this project was worthwhile, maybe I’d say that it’s an exploration of how JavaScript can be used to augment the capabilities of CSS, allowing stylesheet authors to achieve things that would otherwise be impossible.

However others see this project, to me it is a way that I can quickly produce immersive synchronised lyrics using tools and technologies that I’m comfortable with. It precisely satisfies the desires I had when I created it, so I need to be content with that.