Monday, October 7

Animate Images and Videos with curtains.js

 
While browsing the latest award-winning websites, you may notice a lot of fancy image distortion animations or neat 3D effects. Most of them are created with WebGL, an API allowing GPU-accelerated image processing effects and animations. They also tend to use libraries built on top of WebGL such as three.js or pixi.js. Both are very powerful tools to create respectively 2D and 3D scenes.

But, you should keep in mind that those libraries were not originally designed to create slideshows or animate DOM elements. There is a library designed just for that, though, and we’re going to cover how to use it here in this post.

WebGL, CSS Positioning, and Responsiveness

Say you’re working with a library like three.js or pixi.js and you want to use it to create interactions, like mouseover and scroll events on elements. You might run into trouble! How do you position your WebGL elements relative to the document and other DOM elements? How would handle responsiveness?

This is exactly what I had in mind when creating curtains.js.

Curatins.js allows you to create planes containing images and videos (in WebGL we will call them textures) that act like plain HTML elements, with position and size defined by CSS rules. But these planes can be enhanced with the endless possibilities of WebGL and shaders.

Wait, shaders?

Shaders are small programs written in GLSL that will tell your GPU how to render your planes. Knowing how shaders work is mandatory here because this is how we will handle animations. If you’ve never heard of them, you may want to learn the basics first. There are plenty of good websites to start learning them, like The Book of Shaders.

Now that you get the idea, let’s create our first plane!

Setup of a basic plane

To display our first plane, we will need a bit of HTML, CSS, and some JavaScript to create the plane. Then our shaders will animate it.

HTML

The HTML will be really simple here. We will create a <div> that will hold our canvas, and a div that will hold our image.

<body>
  <!-- div that will hold our WebGL canvas -->
  <div id="canvas"></div>

    <!-- div used to create our plane -->
    <div class="plane">

      <!-- image that will be used as a texture by our plane -->
      <img src="path/to/my-image.jpg" />
      
    </div>

</body>

CSS

We will will use CSS to make sure the <div> that wraps the canvas will be bigger than our plane, and apply any size to the plane div. (Our WebGL plane will have the exact same size and positions of this div.)

body {
  /* make the body fit our viewport */
  position: relative;
  width: 100%;
  height: 100vh;
  margin: 0;
  
  /* hide scrollbars */
  overflow: hidden;
}

#canvas {
  /* make the canvas wrapper fit the document */
  position: absolute;
  top: 0;
  right: 0;
  bottom: 0;
  left: 0;
}
.plane {
  /* define the size of your plane */
  width: 80%;
  max-width: 1400px;
  height: 80vh;
  position: relative;
  top: 10vh;
  margin: 0 auto;
}

.plane img {
  /* hide the img element */
  display: none;
}

JavaScript

There’s a bit more work in the JavaScript. We need to instantiate our WebGL context, create a plane with uniform parameters, and use it.

window.onload = function() {
  // pass the id of the div that will wrap the canvas to set up our WebGL context and append the canvas to our wrapper
  var webGLCurtain = new Curtains("canvas");

  // get our plane element
  var planeElement = document.getElementsByClassName("plane")[0];

  // set our initial parameters (basic uniforms)
  var params = {
    vertexShaderID: "plane-vs", // our vertex shader ID
    fragmentShaderID: "plane-fs", // our framgent shader ID
    uniforms: {
      time: {
        name: "uTime", // uniform name that will be passed to our shaders
        type: "1f", // this means our uniform is a float
        value: 0,
      },
    }
  }

  // create our plane mesh
  var plane = webGLCurtain.addPlane(planeElement, params);

  // use the onRender method of our plane fired at each requestAnimationFrame call
  plane.onRender(function() {
    plane.uniforms.time.value++; // update our time uniform value
  });

}

Shaders

We need to write the vertex shader. It won’t be doing much except position our plane based on the model view and projection matrix and pass varyings to the fragment shader:

<!-- vertex shader -->
<script id="plane-vs" type="x-shader/x-vertex">
  #ifdef GL_ES
  precision mediump float;
  #endif

  // those are the mandatory attributes that the lib sets
  attribute vec3 aVertexPosition;
  attribute vec2 aTextureCoord;

  // those are mandatory uniforms that the lib sets and that contain our model view and projection matrix
  uniform mat4 uMVMatrix;
  uniform mat4 uPMatrix;

  // if you want to pass your vertex and texture coords to the fragment shader
  varying vec3 vVertexPosition;
  varying vec2 vTextureCoord;

  void main() {
    // get the vertex position from its attribute
    vec3 vertexPosition = aVertexPosition;
    // set its position based on projection and model view matrix
    gl_Position = uPMatrix * uMVMatrix * vec4(vertexPosition, 1.0);

    // set the varyings
    vTextureCoord = aTextureCoord;
    vVertexPosition = vertexPosition;
  }
</script>

Now our fragment shader. This is where we will add a little displacement effect based on our time uniform and the texture coordinates.

<!-- fragment shader -->
<script id="plane-fs" type="x-shader/x-fragment">
  #ifdef GL_ES
  precision mediump float;
  #endif

  // get our varyings
  varying vec3 vVertexPosition;
  varying vec2 vTextureCoord;

  // the uniform we declared inside our javascript
  uniform float uTime;

  // our texture sampler (this is the lib default name, but it could be changed)
  uniform sampler2D uSampler0;

  void main() {
    // get our texture coords
    vec2 textureCoord = vTextureCoord;

    // displace our pixels along both axis based on our time uniform and texture UVs
    // this will create a kind of water surface effect
    // try to comment a line or change the constants to see how it changes the effect
    // reminder : textures coords are ranging from 0.0 to 1.0 on both axis
    const float PI = 3.141592;

    textureCoord.x += (
      sin(textureCoord.x * 10.0 + ((uTime * (PI / 3.0)) * 0.031))
      + sin(textureCoord.y * 10.0 + ((uTime * (PI / 2.489)) * 0.017))
      ) * 0.0075;

    textureCoord.y += (
      sin(textureCoord.y * 20.0 + ((uTime * (PI / 2.023)) * 0.023))
      + sin(textureCoord.x * 20.0 + ((uTime * (PI / 3.1254)) * 0.037))
      ) * 0.0125;
          
    gl_FragColor = texture2D(uSampler0, textureCoord);
  }
</script>

Et voilà! You’re all done, and if everything went well, you should be seeing something like this.

See the Pen curtains.js basic plane by Martin Laxenaire (@martinlaxenaire) on CodePen.

Adding 3D and interactions

Alright, that’s pretty cool so far, but we started this post talking about 3D and interactions, so let’s look at how we could add those in.

About vertices

To add a 3D effect we would have to change the plane vertices position inside the vertex shader. However in our first example, we did not specify how many vertices our plane should have, so it was created with a default geometry containing six vertices forming two triangles :

In order to get decent 3D animations, we would need more triangles, thus more vertices:

This plane has five segments along its width and five segments along its height. As a result, we have 50 triangles and 150 total vertices.

Refactoring our JavaScript

Fortunately, it is easy to specify our plane definition as it could be set inside our initial parameters.

We are also going to listen to mouse position to add a bit of interaction. To do it properly, we will have to wait for the plane to be ready, convert our mouse document coordinates to our WebGL clip space coordinates and send them to the shaders as a uniform.

// we are using window onload event here but this is not mandatory
window.onload = function() {
  // track the mouse positions to send it to the shaders
  var mousePosition = {
    x: 0,
    y: 0,
  };

  // pass the id of the div that will wrap the canvas to set up our WebGL context and append the canvas to our wrapper
  var webGLCurtain = new Curtains("canvas");

  // get our plane element
  var planeElement = document.getElementsByClassName("plane")[0];

  // set our initial parameters (basic uniforms)
  var params = {
    vertexShaderID: "plane-vs", // our vertex shader ID
    fragmentShaderID: "plane-fs", // our framgent shader ID
    widthSegments: 20,
    heightSegments: 20, // we now have 20*20*6 = 2400 vertices !
    uniforms: {
      time: {
        name: "uTime", // uniform name that will be passed to our shaders
        type: "1f", // this means our uniform is a float
        value: 0,
      },
      mousePosition: { // our mouse position
        name: "uMousePosition",
        type: "2f", // notice this is a length 2 array of floats
        value: [mousePosition.x, mousePosition.y],
      },
      mouseStrength: { // the strength of the effect (we will attenuate it if the mouse stops moving)
        name: "uMouseStrength", // uniform name that will be passed to our shaders
        type: "1f", // this means our uniform is a float
        value: 0,
      },

    }
  }

  // create our plane mesh
  var plane = webGLCurtain.addPlane(planeElement, params);

  // once our plane is ready, we could start listening to mouse/touch events and update its uniforms
  plane.onReady(function() {
    // set a field of view of 35 to exagerate perspective
    // we could have done it directly in the initial params
    plane.setPerspective(35);
    // listen our mouse/touch events on the whole document
    // we will pass the plane as second argument of our function
    // we could be handling multiple planes that way
    document.body.addEventListener("mousemove", function(e) {
      handleMovement(e, plane);
    });
    document.body.addEventListener("touchmove", function(e) {
      handleMovement(e, plane);
    });
  }).onRender(function() {
    // update our time uniform value
    plane.uniforms.time.value++;
    // continually decrease mouse strength
    plane.uniforms.mouseStrength.value = Math.max(0, plane.uniforms.mouseStrength.value - 0.0075);
  });

  // handle the mouse move event
  function handleMovement(e, plane) {
    // touch event
    if(e.targetTouches) {
      mousePosition.x = e.targetTouches[0].clientX;
      mousePosition.y = e.targetTouches[0].clientY;
    }
    // mouse event
    else {
      mousePosition.x = e.clientX;
      mousePosition.y = e.clientY;
    }
    // convert our mouse/touch position to coordinates relative to the vertices of the plane
    var mouseCoords = plane.mouseToPlaneCoords(mousePosition.x, mousePosition.y);
    // update our mouse position uniform
    plane.uniforms.mousePosition.value = [mouseCoords.x, mouseCoords.y];
    
    // reassign mouse strength
    plane.uniforms.mouseStrength.value = 1;
  }

}

Now that our JavaScript is done, we have to rewrite our shaders so that they’ll use our mouse position uniform.

Refactoring the shaders

Let’s look at our vertex shader first. We have three uniforms that we could use for our effect:

  1. the time which is constantly increasing
  2. the mouse position
  3. our mouse strength, which is constantly decreasing until the next mouse move

We will use all three of them to create a kind of 3D ripple effect.

<script id="plane-vs" type="x-shader/x-vertex">
  #ifdef GL_ES
  precision mediump float;
  #endif

  // those are the mandatory attributes that the lib sets
  attribute vec3 aVertexPosition;
  attribute vec2 aTextureCoord;

  // those are mandatory uniforms that the lib sets and that contain our model view and projection matrix
  uniform mat4 uMVMatrix;
  uniform mat4 uPMatrix;

  // our time uniform
  uniform float uTime;

  // our mouse position uniform
  uniform vec2 uMousePosition;

  // our mouse strength
  uniform float uMouseStrength;

  // if you want to pass your vertex and texture coords to the fragment shader
  varying vec3 vVertexPosition;
  varying vec2 vTextureCoord;

  void main() {
    vec3 vertexPosition = aVertexPosition;

    // get the distance between our vertex and the mouse position
    float distanceFromMouse = distance(uMousePosition, vec2(vertexPosition.x, vertexPosition.y));

    // this will define how close the ripples will be from each other. The bigger the number, the more ripples you'll get
    float rippleFactor = 6.0;

    // calculate our ripple effect
    float rippleEffect = cos(rippleFactor * (distanceFromMouse - (uTime / 120.0)));

    // calculate our distortion effect
    float distortionEffect = rippleEffect * uMouseStrength;

    // apply it to our vertex position
    vertexPosition +=  distortionEffect / 15.0;

    gl_Position = uPMatrix * uMVMatrix * vec4(vertexPosition, 1.0);

    // varyings
    vTextureCoord = aTextureCoord;
    vVertexPosition = vertexPosition;
  }
</script>

As for the fragment shader, we are going to keep it simple. We are going to fake lights and shadows based on each vertex position:

<script id="plane-fs" type="x-shader/x-fragment">
  #ifdef GL_ES
  precision mediump float;
  #endif

  // get our varyings
  varying vec3 vVertexPosition;
  varying vec2 vTextureCoord;

  // our texture sampler (this is the lib default name, but it could be changed)
  uniform sampler2D uSampler0;

  void main() {
    // get our texture coords
    vec2 textureCoords = vTextureCoord;

    // apply our texture
    vec4 finalColor = texture2D(uSampler0, textureCoords);

    // fake shadows based on vertex position along Z axis
    finalColor.rgb -= clamp(-vVertexPosition.z, 0.0, 1.0);

    // fake lights based on vertex position along Z axis
    finalColor.rgb += clamp(vVertexPosition.z, 0.0, 1.0);

    // handling premultiplied alpha (useful if we were using a png with transparency)
    finalColor = vec4(finalColor.rgb * finalColor.a, finalColor.a);

    gl_FragColor = finalColor;
  }
</script>

And there you go!

See the Pen curtains.js ripple effect example by Martin Laxenaire (@martinlaxenaire) on CodePen.

With these two simple examples, we’ve seen how to create a plane and interact with it.

Videos and displacement shaders

Our last example will create a basic fullscreen video slideshow using a displacement shader to enhance the transitions.

Displacement shader concept

The displacement shader will create a nice distortion effect. It will be written inside our fragment shader using a grayscale picture and will offset the pixel coordinates of the videos based on the texture RGB values. Here’s the image we will be using:

The effect will be calculated based on each pixel RGB value, with a black pixel being [0, 0, 0] and a white pixel [1, 1, 1] (GLSL equivalent for [255, 255, 255]). To simplify, we will use only the red channel value, as with a grayscale image red, green and blue are always equal.

You can try to create your own grayscale image (it works great with geometric shape ) to get your unique transition effect.

Multiple textures and videos

A plane can have more than one texture simply by adding multiple image tags. This time, instead of images we want to use videos. We just have to replace the <img /> tags with a <video /> one. However there are two things to know when it comes to video:

  • The video will always fit the exact size of the plane, which means your plane has to have the same width/height ratio as your video. This is not a big deal tho because it is easy to handle with CSS.
  • On mobile devices, we can’t autoplay videos without a user gesture, like a click event. It is therefore safer to add a “enter site” button to display and launch our videos.

HTML

The HTML is still pretty straightforward. We will create our canvas div wrapper, our plane div containing the textures and a button to trigger the video autoplay. Just notice the use of the data-sampler attribute on the image and video tags—it will be useful inside our fragment shader.

<body>
  <div id="canvas"></div>
  <!-- this div will handle the fullscreen video sizes and positions -->
  <div class="plane-wrapper">
    <div class="plane">
      <!-- notice here we are using the data-sampler attribute to name our sampler uniforms -->
      <img src="path/to/displacement.jpg" data-sampler="displacement" />
      <video src="path/to/video.mp4" data-sampler="firstTexture"></video>
      <video src="path/to/video-2.mp4" data-sampler="secondTexture"></video>
    </div>
  </div>
    
  <div id="enter-site-wrapper">
    <span id="enter-site">
      Click to enter site
    </span>
  </div>
</body>

CSS

The stylesheet will handle a few things: display the button and hide the canvas before the user has entered the site, size and position our plane-wrapper div to handle fullscreen responsive videos.

@media screen {
    
  body {
    margin: 0;
    font-size: 18px;
    font-family: 'PT Sans', Verdana, sans-serif;
    background: #212121;
    line-height: 1.4;
    height: 100vh;
    width: 100vw;
    overflow: hidden;
  }
    
  /*** canvas ***/
    
  #canvas {
    position: absolute;
    top: 0;
    right: 0;
    bottom: 0;
    left: 0;
    z-index: 10;
    
    /* hide the canvas until the user clicks the button */
    opacity: 0;
    transition: opacity 0.5s ease-in;
  }
  
  /* display the canvas */
  .video-started #canvas {
    opacity: 1;
  }
  
  .plane-wrapper {
    position: absolute;
    
    /* center our plane wrapper */
    left: 50%;
    top: 50%;
    transform: translate(-50%, -50%);
    z-index: 15;
  }
    
  .plane {
    position: absolute;
    top: 0;
    right: 0;
    bottom: 0;
    left: 0;
    
    /* tell the user he can click the plane */
    cursor: pointer;
  }
    
  /* hide the original image and videos */
  .plane img, .plane video {
    display: none;
  }
    
  /* center the button */
  #enter-site-wrapper {
    display: flex;
    justify-content: center;
    align-items: center;
    align-content: center;
    position: absolute;
    top: 0;
    right: 0;
    bottom: 0;
    left: 0;
    z-index: 30;
  
    /* hide the button until everything is ready */
    opacity: 0;
    transition: opacity 0.5s ease-in;
  }
  
  /* show the button */
  .curtains-ready #enter-site-wrapper {
    opacity: 1;
  }
    
  /* hide the button after the click event */
  .curtains-ready.video-started #enter-site-wrapper {
    opacity: 0;
    pointer-events: none;
  }
    
  #enter-site {
    padding: 20px;
    color: white;
    background: #ee6557;
    max-width: 200px;
    text-align: center;
    cursor: pointer;
  }

}

/* fullscreen video responsive */
@media screen and (max-aspect-ratio: 1920/1080) {
  .plane-wrapper {
    height: 100vh;
    width: 177vh;
  }
}

@media screen and (min-aspect-ratio: 1920/1080) {
  .plane-wrapper {
    width: 100vw;
    height: 56.25vw;
  }
}

JavaScript

As for the JavaScript, we will go like this:

  • Set a couple variables to store our slideshow state
  • Create the Curtains object and add the plane to it
  • When the plane is ready, listen to a click event to start our videos playback (notice the use of the playVideos() method). Add another click event to switch between the two videos.
  • Update our transition timer uniform inside the onRender() method
window.onload = function() {

  // here we will handle which texture is visible and the timer to transition between images
  var activeTexture = 1;
  var transitionTimer = 0;
  
  // set up our WebGL context and append the canvas to our wrapper
  var webGLCurtain = new Curtains("canvas");
  
  // get our plane element
  var planeElements = document.getElementsByClassName("plane");
  
  // some basic parameters
  var params = {
    vertexShaderID: "plane-vs",
    fragmentShaderID: "plane-fs",
    imageCover: false, // our displacement texture has to fit the plane
    uniforms: {
      transitionTimer: {
        name: "uTransitionTimer",
        type: "1f",
        value: 0,
      },
    },
  }
    
  var plane = webGLCurtain.addPlane(planeElements[0], params);
    
  // create our plane
  plane.onReady(function() {
    // display the button
    document.body.classList.add("curtains-ready");
  
    // when our plane is ready we add a click event listener that will switch the active texture value
    planeElements[0].addEventListener("click", function() {
      if(activeTexture == 1) {
        activeTexture = 2;
      }
      else {
        activeTexture = 1;
      }
    });
    
    // click to play the videos
    document.getElementById("enter-site").addEventListener("click", function() {
      // display canvas and hide the button
      document.body.classList.add("video-started");
      
      // play our videos
      plane.playVideos();
    }, false);
    
  }).onRender(function() {
    // increase or decrease our timer based on the active texture value
    // at 60fps this should last one second
    if(activeTexture == 2) {
      transitionTimer = Math.min(60, transitionTimer + 1);
    }
    else {
      transitionTimer = Math.max(0, transitionTimer - 1);
    }
    // update our transition timer uniform
    plane.uniforms.transitionTimer.value = transitionTimer;
  });
}

Shaders

This is where all the magic will occur. Like in our first example, the vertex shader won’t do much and you’ll have to focus on the fragment shader that will create a “dive in” effect:

<script id="plane-vs" type="x-shader/x-vertex">
  #ifdef GL_ES
  precision mediump float;
  #endif

  // default mandatory variables
  attribute vec3 aVertexPosition;
  attribute vec2 aTextureCoord;

  uniform mat4 uMVMatrix;
  uniform mat4 uPMatrix;

  // varyings
  varying vec3 vVertexPosition;
  varying vec2 vTextureCoord;

  // custom uniforms
  uniform float uTransitionTimer;

  void main() {

    vec3 vertexPosition = aVertexPosition;

    gl_Position = uPMatrix * uMVMatrix * vec4(vertexPosition, 1.0);

    // varyings
    vTextureCoord = aTextureCoord;
    vVertexPosition = vertexPosition;
  }
</script>

<script id="plane-fs" type="x-shader/x-fragment">
  #ifdef GL_ES
  precision mediump float;
  #endif

  varying vec3 vVertexPosition;
  varying vec2 vTextureCoord;

  // custom uniforms
  uniform float uTransitionTimer;

  // our textures samplers
  // notice how it matches our data-sampler attributes
  uniform sampler2D firstTexture;
  uniform sampler2D secondTexture;
  uniform sampler2D displacement;

  void main( void ) {
    // our texture coords
    vec2 textureCoords = vec2(vTextureCoord.x, vTextureCoord.y);

    // our displacement texture
    vec4 displacementTexture = texture2D(displacement, textureCoords);

    // our displacement factor is a float varying from 1 to 0 based on the timer
    float displacementFactor = 1.0 - (cos(uTransitionTimer / (60.0 / 3.141592)) + 1.0) / 2.0;

    // the effect factor will tell which way we want to displace our pixels
    // the farther from the center of the videos, the stronger it will be
    vec2 effectFactor = vec2((textureCoords.x - 0.5) * 0.75, (textureCoords.y - 0.5) * 0.75);

    // calculate our displaced coordinates to our first video
    vec2 firstDisplacementCoords = vec2(textureCoords.x - displacementFactor * (displacementTexture.r * effectFactor.x), textureCoords.y - displacementFactor * (displacementTexture.r * effectFactor.y));
    // opposite displacement effect on the second video
    vec2 secondDisplacementCoords = vec2(textureCoords.x - (1.0 - displacementFactor) * (displacementTexture.r * effectFactor.x), textureCoords.y - (1.0 - displacementFactor) * (displacementTexture.r * effectFactor.y));

    // apply the textures
    vec4 firstDistortedColor = texture2D(firstTexture, firstDisplacementCoords);
    vec4 secondDistortedColor = texture2D(secondTexture, secondDisplacementCoords);

    // blend both textures based on our displacement factor
    vec4 finalColor = mix(firstDistortedColor, secondDistortedColor, displacementFactor);

    // handling premultiplied alpha
    finalColor = vec4(finalColor.rgb * finalColor.a, finalColor.a);

    // apply our shader
    gl_FragColor = finalColor;
  }
</script>

Here’s our little video slideshow with a cool transition effect:

See the Pen curtains.js video slideshow by Martin Laxenaire (@martinlaxenaire) on CodePen.

 

This example is a great way to show you how to create a slideshow with curtains.js: you might want to use images instead of videos, change the displacement texture, modify the fragment shader or even add more slides…

Going deeper

We’ve just scraped the surface of what’s possible with curtains.js. You could try to create multiple planes with a cool mouse over effect for your article thumbs for example. The possibilities are almost endless.

If you want to see more examples covering all those basics usages, you can check the library website or the GitHub repo.

Source: CSS-tricks.com

0 0 votes
Article Rating
Subscribe
Notify of
guest

0 Comments
Inline Feedbacks
View all comments
0
Would love your thoughts, please comment.x
()
x