Simon: It's always a hassle to determine, okay, where do I have to go? Where I am, where I'm at at the moment. And so with this movement API implemented in Nut.js, you basically are able to just say, "Move my mouse left by a hundred pixels."
Andrew: Hello, welcome to the devtools.fm podcast. This is a podcast about developer tools and the people who make them I'm Andrew, and is my cohost.
Justin: Hi everyone today. We're joined by Simon, who is the creator of Nut.js. Nut.js is a node library for automating cross-platform native UI. Simon, is there anything you'd like to tell our audience about yourself?
Simon: Not in particular. So I'm, I'm pretty excited to be on the show. Thanks for having me!
Justin: Yeah, we're to have you here!
Simon: And I guess that's it.
[00:00:54] Nut.js
Justin: Well, maybe to start off with, can you just tell us a little bit about what inspired you to create, Nut.js and maybe if you want to give a better description than what I did, maybe we can start off there too.
Simon: So in general, I looked into the topic of desktop automation using JavaScript. Yeah, around July, 2018. And back at that time, I was pretty unhappy with the results I found because, well, I wanted to, to have something which lasts for a long time or a longer time, and especially in the node ecosystem, that's a bit hard.
And so pretty much every library I took a look at was either unmaintained had failing CI or yeah Was pretty much dead And after awhile I decided that well I will take shot at it and started working on Nut.js, and did a little prototyping And when I realized that this could actually work the way I want it I invested more and more time in it And well here we are over three years later and still going
Justin: That's how it goes. That's how it goes. when you, when you set out to build the library, was there a particular problem that you were trying to solve? Was there like something very specific that you're trying to work through.
Simon: So basically the main goal I tried to achieve was using it for integration with the web automation tool So basically the the overall project which which kind of came together was a combination of a browser automation combined with desktop automation and all the tools I had a look at where basically lacking the the whole image recognition stuff . So finding a library which allows you to automate your mouse or your keyboard input was pretty straightforward There are actually several out there besides the fact that they are maybe unmaintained but having having a tool which allows you to also navigate your desktop by providing template images was nonexistent and well that's that's a topic I picked up and decided to implement it myself
Andrew: So does that mean you use Nut.js to do your full automation? You don't rely on something like playwright or something that's integrated with the browser. You just. Really more automate the computer, running the.
Simon: So Nut.js itself is basically more targeting the whole PC itself. So you're, you're able to, to simulate keyboard inputs, you're able to navigate your mouse around your desktop and as a. It's possible to provide template images, and then it would scan your desktop and say, okay, I located it in this area.
I will move the mouse to, to this area. And your browser automation was done back in the days with selenium and the combination of these basically evolved into a separate framework.
Justin: Yeah. actually stumbled across, uh nut.js doing a funny sort of thing. One of my cousins passed away in the summer of 2020. He and I used to play runescape a lot when we were younger. So I got back into just for nostalgia purposes just playing some runescape.
And if anyone has ever played that it's a browser-based game. It used to be like a javlett app that ran in the back browser they've since updated it. But it's a it's, it's a game that's like designed to take as much time as possible. Like it just takes a ton of time to do anything. It's a point and click type game.
And I was like, well, it'd be fun to, but this it's like, I want to like make my own bots So I need something to control the keyboard. I need something to control the mouse and I need some way of like detecting things in the screen. And I went about like, just looking for node libraries to do this. I ran across something called robot JS, which like helps control the mouse and keyboard.
And then I was really having a hard time trying to find something that would do the display and I'd run across this like open CV library, open CV is a open computer vision library that like helps you distinguish things on the screen. And all that looked really interesting, but it was like hard to mesh everything together and things are unmaintained and it was like, it was kind of crappy.
And then I come across nut, which like did all the things that it provided this, like, unified wrapper for everything and I was like, "Oh, this is really cool."
So I was able to like write my own bot in like an afternoon. And it worked really well. And it was, it was pretty fun.
Simon: That's cool to hear. And actually you're not the only one who was trying to bot runescape. So,
Andrew: With Nut.js specifically?
Simon: Yes. So, um, I'm, um, I'm running a discord server for the, for the little nut.js community work. And I think if I remember correctly, they were an hour, at least five people asking me whether I could help them botting rune scape.
Justin: Uh,
Simon: And, um, I'm not sure I don't want to make any, any false statements, but I guess there's somebody making a profit out of it.
Justin: It'd be, could be, it's definitely getting their terms of service. So if your botting you're gonna get banned, but it is also just like a fun technical challenge, because there are like specific things that you have to be able to do. It's like detecting where you are in the world and being able to like, Control inputs and stuff.
Anyway, it was, it was really fun. And I think the abstractions that nut provides, it's like being able to really easily capture a piece of the screen and use that as like, Hey, if this piece of the screen exists, then you know, it's in this state, wait for this thing to exist. That was really refreshingly simple.
And I think that keeps me into the applications, you know, from a broader context it's like, especially desktop automated testing.
Simon: Yeah, that was actually also one of the main goals I try to achieve because it's with many, many automation libraries you're running into the truck problem that, well, it's possible to do the two things, but everyone will write his own utility functions to make the API enjoyable. And so one of my main goals was to, or at least trying to design an API, which is basically fun to use.
It gets you off the ground in like a minute or so. So my goal was you look at an example, you grasp the concept and then you're good to go. So I hope that this works.
Andrew: Yeah, it's it seems like an example of finding the right primitive to make, make the program easy to use. And the right primitive in this case just happens to be looking at the screen and recognizing something's on the screen. So that's pretty cool.
Simon: and also with, with movement because, that's, that's something I implemented only after awhile because I noticed myself that I want an easier way to navigate my screen. It's always a hassle to determine, okay, where do I have to go? Where I am, where I'm at at the moment. And so with this movement, API implemented in Nut.js, you basically are able to just say, "Move my mouse left by a hundred pixels." For example, or in combination with the template image matching, it's possible to say, move my mouse straight to the center of the image I'm providing you. And so once the image is located, it will move to the mouse on a straight line to the center of this image provided.
So it works by composing multiple really simple functions. Which gives you a pretty powerful and expressive API. What, what's something I'm really enjoying.
Justin: There's a, there's kind of a hard part of automating, like automating desktop interactions where the thing that you're interacting with, like say you want to like click an icon? Well, the icon might be transparent. The background might be different. So like selecting that particular icon might be different in different situations.
So that, that can be a particularly challenging thing where like a flat screenshot might not be sufficient. So I know that you integrated with the open CV library and you were using that in, under the hood. Is it still all just like flat image matching or do you have any more sort of like robust pattern matching that, that the API tap into.
Simon: So at the moment it's a really simple template matching, but maybe as a little sneak peek, I'm currently working on a new major release. And one of the topics I want to address is making it easier to use Nut.js in terms of dependencies. And the other thing is that I definitely want to improve the image matching part because it, it still has its flaws.
So, as you just said, for us humans, it's pretty easy to say that's definitely the icon I'm looking for, but expressing it in code is not that simple. And so I'm currently looking into ways to make image matching more reliable, even with scaling applied, maybe rotation applied, stuff like that.
Andrew: Does Nut.js work in CI
Simon: Um, kind of, you still have to provide a virtual display, but for example, my integration tests on GitHub are all running, using, for example, a virtual frame buffer on a Linux host. Nut.js has has a full, a suit of full end-to-end tests. And these end to end tests are running inside a Docker container, which provides a virtual display.
So it's always putting up the exact same desktop environment. And then I'm dragging icons around that it is verifying their location and stuff like that, but it's doable to run it on CI in the background using for example a docker container.
Justin: That's pretty cool. That'd be a really interesting way to automate like desktop application testing. Certainly interesting.
Simon: for example, I know from some people that they are using Nut.js to actually do application testing, but in an environment where they do not have direct access to the system. So basically they are connected to via a remote connection. And they only have to image information. So they use Nut.js and use the image matching capabilities to basically verified that the UI they are currently reworking is like, they would expect it
Justin: Oh, that's really interesting.
Simon: on the target system.
[00:12:01] Native Dependencies
Justin: I want to talk a little bit about the native dependency in nut. So you have this library called lib nut. It's written in C, C plus plus. So it implements a lot of the like underlying cross system functionality, like key press detection, mouse control, you know, screen detection, et cetera.
Can you talk, can you just talk a little bit about what it was like developing that library? I mean, I know a lot of people who listen to this probably haven't taken the task of like building a native dependency that works with node. And I don't think it's always the most straightforward thing to do.
Simon: Yes, in general, Nut.js has actually two native libraries because this whole open CV part is a native node addon as well. But for the lib nut part, so essentially lib-nut is a fork of robot.js.. The node framework you mentioned earlier, but unfortunately this framework is no longer maintained.
So back in the days I decided to fork it and maintain it myself. That's how lib-nut actually came into existence. Over time I changed quite a lot in the, in the, in the project. The, yeah, I guess of the most prominent difference to usual node development when it comes to native addons is you're dealing with a whole different environment in regards to: what kind of build system are you using?
What kind of language are you using? How do you distribute a native add on? Because in the end, a native add on is a binary. You have to, you have to build, uh, an add on for Mac OS, for Linux and for windows. So there, you have to also think about how do I manage my packages, that I can provide three platforms with as little overhead as possible.
And so of the changes I applied were basically all learned the hard way, because there are also two different ways to develop native addons. So the initial version or the, the, the initial fork of Robot.js used a tool called native abstractions for node. And the problem with these native abstractions for node is that these binaries which are generated are also dependent, dependent on the node js version you're building them for which means on every new major release, you will have to run a new build to provide new bindings. And the open CV on I'm using is currently still using this tool because I just haven't had the time to get rid of it because open CV for node js. So that's the library which provides these bindings is huge.
On the other hand, it's the most complete open CV binding you find fornode js, but, well, that's a different story. With all these, with all these problems I was facing, I, first of all, decided that I would want to use a build system. I already knew. So I ditched node-gyp and switched over to using CMake JS.
So basically just the JavaScript wrapper around CMake. And the second major change I applied was that I did no longer use these natives abstractions for node but with node 10, they released the so-called node API, which is another way to, to build native node addons. But in contrast to native abstractions for node, you no longer depend on the particular node version.
So now it's possible to build it for let's say node 12, and you can still use it with node 16 without having to recompile anything. So that was a huge win for me. And also, yeah. Lowered the amount of work I had to take care of because, well, I will just build it one time and you can use it with any node version from 12 upwards you want.
Yeah, that's basically it with these major problems out of the way, it was easily possible to get, to just continue working on it. And in terms of distribution, that's also something I solved in a pretty, I would say lazy man's way because I'm, I'm only published. So I'm basically just patching the package.json and my CI job.
The CI job is running on Linux, Mac OS and windows using github actions. And these three published packages are then all incorporated in one, I would say meta package. There's a node package, which is called for example, lib-nut win 32, for windows. And this package is a dependency for the overall lib-nut
and depending on which target platform you're installing lip nut, it will basically re-export the bindings for the respective target platform. That's pretty straightforward way to do it. And it works seamlessly.
Justin: Yeah, that's really interesting. I I know that yarn, the package manager had just released a major version where in older versions they had an issue where if you had a cross-platform dependency like this, they would download all the binaries for all the platforms, even if you didn't need it. And I, I forget, I forget exactly why they did that, but there was some complexities and, and like system analysis or something that they had to do, but they solved, uh, in version three.
So just, I don't know, this just made me think of that.
Simon: Just for dealing with, with binary items, there are actually quite a few packages out there. I guess the most prominent at the moment is called prebuildify. And prebuildify if I remember correctly does Something like, you're just like you just mentioned. So it, it basically keeps or contains all the binaries for all the platforms. So that the tool I'm using for the open CV bindings is using a bit of a different approach.
So it basically during CI it generates native bindings for every platform and every node version I want to use it for and every electron version. And, once the binary is built, it uploads it to the GitHub release. So the GitHub release currently has around 30 binaries attached. And, when you install the package it will check:. Okay. Which target platform am I running on? Which node version am I running? And then grabbed the respective binary from the GitHub release and unpack it in your node modules folder.
Andrew: Yeah, for, for a long time, I didn't know that node packages could work like that. I thought everything was just JavaScript. And I was helping out on, uh, a project called package PKG and they do the same thing where they have this like separate project that has GitHub actions that builds their custom version of node or like an add on for it.
And it's, it's like a, a blindingly confusing world for me. I think most people, when they look at node, wouldn't, wouldn't even realize that you could do stuff like this inside of node.
Simon: Yeah. So most of the time, when, when talking to people about the stuff I do in my spare time, they were like, you're crazy, man, why don't you do this?
Andrew: Yeah. Subjecting yourself to that pain.
Simon: But actually now it's just a pretty straightforward thing to do because I have all the setup. My development workflow basically consists of, okay, I have finished a new feature. I will just merge the feature branch into develop. And then from dependency to dependency upwards, it will just trigger a new build.
It will, once it's completed, triggered the next dependency, which builds and so on and so on. And in the end, I have a ton of new snapshot releases on NPM. It was just a finger snip. So it, it definitely pays off to invest in these kinds of things.
Andrew: Yeah. The, the, those type of workflows that get have actions has enabled are just like awesome. Like being able to just run on a windows machine with like little to no effort is A+.
Simon: yeah, that's definitely without GitHub actions I wouldn't be able to develop nut.js because I'm only running a Mac and it's not feasible to build all the binaries on my, on my machine.
Justin: There are some languages that, that do a pretty good job of, building cross-platform binaries. So like Zig and go, I think are two in particular that do a decent job of doing, uh, cross-platform compilations. I don't exactly know how that works. I'm not sure like what the exact details are there, but I just saw a tweet today.
I think that was about. the CLI framework for elixir, where they use Zig under the hood to make it where you can compile across platforms, CLI framework in elixir for just like any, any system or whatever. I don't know. Just seems like magic to me.
Simon: I only read about it. So, especially from, from go people who are basically always telling me that, oh yeah, look, I will just change my target architecture. And then I run go build and it will just spit out a binary for a different target platform. But so far I haven't looked into cross-platform builds.
That's so. It would reduce the amount of CI work I have to do, or the amount of work I'm using, but it's simply you haven't found the time to invest.
Justin: It seems like a non-trivial effort.
Simon: Most likely!
Andrew: So we talked a little bit about, two examples of things you've seen automated with Nut.js, one being your like test automation and the other being, apparently RuneScape's very popular, but, Some other cool examples you've seen out there in the wild and on your discord of what things people are doing with Nut.js?
Simon: So I guess one of the things I'm most proud of, I'm not sure whether it's okay to, like, I brag about it in here, but I'm looking at the projects on GitHub, which are using Nut.js, one of them is the vs code extension tester developed by Red hat. When I noticed that Red Hat had started using my package, I was, it, it kind of blew my mind because Red Hat was like one of the first companies which brought me in touch with, with like open source software using their Linux distribution now almost 20 years ago. And that, yeah, that was pretty amazing. Yeah, given, given the fact that they are developing like this vs code extension tester, which is used in obviously pretty much VSCode extensions.
Now the software I'm building ends up being used in projects, developed by companies like Microsoft or Intel or SUSE, which is another Linux distribution I used in my really, really early days of Linux. So. That's pretty amazing and all just developed by one single guy is sitting in like the middle of nowhere in Southern Bavaria.
Andrew: Yeah, it's, it's pretty cool that something you just like made in your free time just to scratch an itch is now being used to power probably like thousands of projects. If it's used in the vs code extension tester.
Simon: definitely.
Justin: Yeah, it's definitely the beauty of open source.
Simon: Yeah. I just want it to say exactly that that makes every open-source enthusiasts heartbeat.
Andrew: It can also be pretty stressful in that now you're depended on by thousands and thousands of people. Got to step lightly.
Simon: Yeah. It, it changes a bit. That's definitely true. So seeing that there are a lot of people using stuff you're building puts quite a lot of pressure on you, but at, at the moment it's still a bearable.
Justin: So I do have to ask where did the name. What, what inspired the naming of, of nuts.js.
Simon: That's a pretty lame story because I started. Developing Nut.js with the prototype name of native UI testing. Then it evolved into native UI toolkit. And then I looked at it and said, the first three letters are basically nut okay. Then I, one evening I sat down and started drawing this Walnut logo on my iPad.
And well, I just went with it.
Justin: Nice. Nice.
Andrew: That's awesome that you you made the logo too. We made the logo for our podcasts and I'm proud of that.
Simon: I even have my own stickers on me
Justin: Oh, there you go. We still have to do swag.
Simon: that's th that's the benefit of having your own logo?
Justin: sure.
[00:25:28] Automate all the Things
Justin: So you obviously have a passion for, for automation. What are other things that have you, have you like tried just automating other things in your life. Do you like use automation and other places?
Simon: Not in regards to home automation because when it comes to this, I'm relying on boring, old mechanical stuff. I do not want to wire up my oven to, to my wifi because that makes no sense in my opinion, but yeah, with regards of software and trying to automate very securely, what's possible.
So in, for example a few years ago I noticed that I haven't updated my CV in quite awhile. And once I finished university, I was in the process of sending out applications. And then I noticed, "Oh damn! I have to update my CV and I haven't done it in like six years." A ton of stuff happened in between. I should do this on a regular basis. And that's when I decided that the easiest way for me to keep track of what I'm doing was to put it in the git repository.
So now I have my CV in a latex file in the case repository. And then the next thing was, I do not want to build the PDF file every time it applies some changes. So now I hooked up a build job to my repository, which spits out a PDF using latex and K. And to make the whole story and then decided that it would also be nice to once I validated that the output does not look screwed, I could just publish it.
And so now I'm verifying using IM image matching that some of the key points in this PDF, or I know that they should be at the exact same position on every page, otherwise the lay out is screwed. Once that is in place, I can just go ahead and publish my PDF file to my server, and then it's available to the public.
So these are really the kind of things I am enjoying in my spare time. Kind of strange for other peoples, but that's yeah, it just gives me joy.
Andrew: Yeah, it's, it's always really fun to be able to find a problem that you can solve with code that you might not otherwise be able to solve. So like in your case, it was updating your resume. But my, my favorite story to share from my past is I wrote a script to help me win Coachella tickets. So the a local radio station was running a campaign where they in the morning announced three artists that they were going to play back to back that were all playing at Coachella.
And then you'd have to call in, , and be like the seventh caller. And then you got to Coachella tickets. And I was like, wow, that sounds like a programming problem to me. So like I, like after work, I went straight to my computer and was able to use. I use Twilio to send out text messages to people.
And then I I hacked into Shazam and where they store the notifications on my computer. And I read from the SQL Lite database where it's all stored. And I was able to read which which artists had been recognized from the playing radio stream on my computer. And then it would blast out a text to me and all my friends and I was able to win four, four Coachella tickets doing that. But that's the story I tell when I want, uh, I tell people that programming is useful outside of making programs.
Simon: Former colleague of mine did quite this exact same for the football world cup in 2008, but it was in Germany, Austria, and Switzerland for remember correctly.
Andrew: Yeah. He won a radio contest?
Simon: Yeah. So the, and the, and the world cup before if I remember correctly and I'm not sure whether it's okay to tell this story, he basically paid his son to get up whenever you, his ticket alarm went off and then his son had to to go to the computer and basically claim the ticket and in the neck, during the next event, he then automated also.
That piece as well,
Andrew: Yeah. That's a different form of automation using your children as, as workers.
Simon: as long as it works.
Justin: That's the, that's the type of automation that comes with the experience that you like. I know how much work this is. I'm just going to get somebody do this for me. The next tier of automation.
[00:30:14] The Future of Web
Justin: Well, for speaking about the next tier this is something that we've, we've talked about, uh, to different people in the past.
And it's sort of a generic question, but what we'll ask you to What do you think the future of the web will look like? What do you like when you envision that? What does it look like to you?
Simon: So I'm, I would say both pretty confident about the future of the web, because especially seeing stuff like Photoshop Entering the web or for example, applications like Figma. So Figma runs in the browser just as smooth as it would on a desktop. So I'm pretty sure that the web will in the near future evolve into the universal platform.
So I'm betting on that. Maybe you would need a pretty expensive MacBook to just run your browser because every, everything else is on the browser, but the things we are capable of right now, performing in the browser are already pretty impressive. And I guessed that this will only increase in the future.
And on the other hand, I'm a bit hesitant because it basically boils down to the three big companies steering the web because there's only Chrome Firefox and safari left. Safari being like the, the other odd little brother who wants to, who wants to play as well. But in a sense, so Firefox and Chrome, basically you are deciding over the web and that's maybe not the best thing to do because my opinion, something like your, some something that big as the worldwide web should definitely have more people to decide upon its, its future.
Justin: Yeah, definitely. It's the, the leaks from the, uh, Google's monopoly antitrust case are showing that. I mean, even though. You know, people from the Chrome team have expressed Goodwill and intent. It looks, it looks dicey. I mean, it definitely looks like Google is doing what I guess makes monetary sense for them and using Chrome as a sort of lever to operate and gain more information, you know, get more analytics for ads ultimately.
So yeah, I definitely, I definitely wish it was more competition in the space, but you know, building a browser is hard. So, you know, it's, it's a
Simon: Yeah, definitely.
Justin: problem. And I think in some ways we've, we've all benefited from the rise of chromium and the, you know, It's pushed the state of the web for it a lot, but you know, it's not without its cost.
So yeah.
Andrew: To touch on what you said about needing a powerful Mac book. There are projects out right now that do aim to, to like solve that. Have you heard of mighty browser? Mighty browser is literally your browser in the cloud. So, instead of running the code on your machine, in your browser locally, it's all run on really powerful machines in the cloud.
And then the updates are just streamed to your browser. So you have a potentially a netbook and still be running Figma at 60 frames per second.
Simon: That's basically the these remote gaming services from, from, Nvidia and so
Andrew: Very similar.
Simon: term turned into, into browser. Yeah. Makes sense.
[00:34:10] Wedding App
Andrew: So last question, I was browsing your website and your blog, and I noticed that you built a progressive web app for your wedding. Uh I'm I'm going to be getting married here in the next few years. And my family was joking that I would probably start building my own wedding website too. So why, why did you do that?
Did it go well, did people actually use it? Do you still go back to it? Is it like a little memory and time that you can go visit?
Simon: yeah. So why did I build it? I guess? 'cause I equally love building stuff as I like automating stuff. That was just my, I thought about what we could need for our wedding. And one of the central problems is always that you have to get information out to your guests. And so I decided, well, the most, the easiest way would be to just slam it on a website.
And then I started thinking, "Hmm. But if I'm building a website and maybe I can take it a step further." And the one thing I was was really sensitive about is that when it comes to private events, such like my wedding, I'm really conservative when it comes to privacy. So I do not want any photos of my wedding flying around Facebook, for example.
And that's when I came up with this idea that. One of the coolest things you actually do at a wedding is you express your gratitude towards the yeah. The, the, I don't know, the English term. So towards the bride for example, You wouldn't do this necessarily in public because that's also something really, really personal, or maybe you want to share your own pictures from, from this particular wedding to give the bride to you like your, your, your, personal view on this, on the very very special day.
And so I came up with this idea that I would want to have a progressive web app, which allows our guests to upload either little messages or pictures. But with this little twist that every single guest was only able to see it's his own content. So everyone was able to post pictures and it didn't matter which kind of pictures, because he was the only one who would see it, despite me at my wife, because in our stream all the all the combined pictures and messages from all of our guests were assembled.
So we were able to follow along the stream of pictures of all of our friends while everyone was basically yeah. Preserving his own, his own privacy. And yeah, I'm still coming back to this app every once in awhile because it's sweet memories.
Andrew: Yeah, I might fork you out for my wedding.
Simon: So I can definitely recommend it because it's, it was a really, really nice experience because you could really feel that people were not concerned about "Can I post this? What do other people think? Who can see this might've get in trouble?" It was just like, Do you know, I'm super happy to be here and I love you guys.
I have a wonderful night and stuff like that. It's that's yeah, it's on a different level. If you can make sure that it's really something you've share in privacy
And we just had a little QR code inside this little, little, the table table info card. Everybody would just scan it with their cell phones and off they go.
Andrew: Cool. I think with that, we can start heading into the tool tips.
[00:38:08] Tooltips
Andrew: So my first tool tip of the week is updated docs for react.
It's hooks have been out for almost three years now.
Justin: crazy. Crazy.
Andrew: Crazy it's been out so long and in the react docs, there's basically just one page that tells you how to use hooks and the rest of the docs tell you to use class components and to use basically the legacy react API. So over probably the course of the last year or two Rachel Nabors, I think? One of the people on the react documentation team and a bunch of open source contributors have been building this new react docs website, which looks nice and bubbly and react-y, but it also has a bunch of tutorials that are interactive and walk you through learning react and everything is hooks focused now. So it's a lot easier to wrap your head around what's going on and for new people in the ecosystem to come and start learning. So if you haven't checked out react's docs before, now's the time.
Justin: That's great. And yeah, the website is beta.reactjs.org at the moment .
Andrew: It will be in the show notes.
Justin: Indeed. And I will not forget to publish the sheriffs.
Andrew: We can only hope.
Justin: Indeed.
So my share this week is this library called Figma export by GitHub, user marco-montalbino we'll, we'll go with that. So, this is, uh, an interesting thing that I ran into at work. We have a set of icons stored in Figma, and I want to automatically export them so that I don't have to manually do this. Figma has a really great API, but it's not something that I wanted to faff about with.
So this library, Figma export gives you a CLI and a set of libraries that let you export things from Figma and different ways. So you just go to Figma, you get an API key and then you basically put in the file ID, which is part of the URL. And then what page that you want, the icons or whatever, you're exporting to be exported.
And yeah, you wire it all up, run the CLI command and it'll just write everything out. So right now I'm working on a PR to have all of our. SVGs exported as a, a react component. And it like works really well. It's like super good.
There's also, yeah, there's also like the ability to export styles as like CSS variables and stuff.
So potentially like doing design tokens if you're building a design system. So yeah, a lot of cool stuff.
Andrew: Yeah, It would be cool. I dunno if it already does this, but if it runs SVGO on these, because Icons are rarely ever optimized and it's always good to run them through SVGO.
Justin: So it has two types of plugins for this. It hasoutputters, which like change the output format. So you can do like rec components and stuff like that. And it transformers. So you and their one transformer that they have right now is for SVGO.
So yeah, you can
Andrew: Oh, there you go.
Justin: Okay. The optimizations that way.
Andrew: Yeah. This seems like a no brainer way to, to maintain your design tokens.
Justin: There's an interesting question about like, I mean, at the point that you set up a script like this, you, you put an implicit dependency on things like how you name components and Figma, or how you name icons and Figma, and then what they're represented in as code. And, you know, there's an argument to be made that that's a fragile coupling because, you know, it's, it's hard to know if you've made breaking changes or whatever.
But there's also something to be said about like things like falling out of sync and how much of a, just an absolute pain that can be. So, I don't know. I want to think about it a little bit more and figure out how to make this more of a scalable process. But for right now, this is so, so, so much better than like manually exporting every single stinking icon.
Can't tell you how long I thinks it's awful.
Simon: First make it work, then
Justin: Yeah, Exactly.
Exactly.
Andrew: Yeah, I I've done. I've gone through the process of teaching our designers back Intuit about semver and getting them to not make breaking changes. That's that's a challenge. Definitely an education issue.
Justin: Yeah, I think it's one of those things. When, when you're developing a design system that you just have to come to an understanding of like the things that are important, the things that are changeable, you know, that's part of the work.
Andrew: Next up, we have lerna audit.
Simon: Yes. So that's my first tool tip I brought today and that's a little, little bit, a little helper, NPM utility package called lerna audit. And it's developed by a former colleague and friend of mine called Tim. Shoutouts to Tim! And it basically helps you. When you're managing amonorepo using lerna.
And in case you want to run an NPM audit or an NPM audit fix on this monorepo you will run into trouble because NPM fix will try to download your inter package dependencies from the registry where they are not yet and lerna audit is basically a little wrapper tooling around it, which takes care of managing managing all the package dependencies. So you can successfully run, lerna audit on every of your sub packages and later on putting everything back in place. So it saved us a ton of time and in a, in one of our projects and it's enjoying full to use. So I can only recommend it
Justin: Yeah, that's awesome!
Simon: Except that you're using yarn and their workspaces because I heard that this should not be such a trouble.
Andrew: Even though yarn and now MPM or implementing workspaces, I still find myself wanting to use lerna in in most my monorepo projects. So it's, it's good to have tools like this.
And so since we've been talking a lot about automation, I thought I'd share the automation framework I've been using to do test the share pages at descript. And we landed on playwright cause playwright it, the cool thing about it to me is that you can test almost any targets so you can test on safari through web kit.
You can test on Firefox. Of course you can test on Chrome, but you can also test on electron since descript is an electron app. It's good to have the option to be able to use the same testing framework for our electron app. And then one cool thing that the last time I looked at playwright, they didn't have is they have this playwright test package, which is a test runner that's built around playwright.
So it works really well. It can run things in parallel. It has tests and expect just like any other. Framework you might have, and I've been enjoying using it.
Justin: People might be familiar with puppeteer. Puppeteer is kind of the older version of this. Actually, a lot of the team that works on playwright used to work on puppeteer. There were some like, sort of drama around that. I'm not sure exactly what happened, but puppeteer was, I believe a Google project. And then a lot of the devs that worked on that project, like abandoned ship and it went to Microsoft and restarted the team and just used a lot of the lessons that they gained from puppeteer and applied them to, to playwright.
Andrew: It was very little set up to get everything going. Like in my initial stab at this, I went through jest playwright and it was kinda kind of hard to juggle and it was an extra layer of abstraction that made things kind of harder to understand. This, it's, it's very simple. It doesn't require you messing with like any TypeScript, configs or Babel configs or anything.
It just works. So I highly recommend.
Simon: That's definitely something on my list to check out. Currently I'm only using Cyprus, but it would be nice to get experience with playwright as well.
Andrew: Yeah, you don't, you don't get the nice UI that you get with Cyprus. But still works really well.
Justin: Okay. So my second tool tip of the day is, and I got one last tip, I guess, is an article and a tool from Anthony Fu. So if you're not familiar with Anthony Fu's work, he is one of the developers behind windi CSS, which is a tailwind like CSS thing. I think it inspired the, the Jit a mode in tailwind.
So he's working on a new project now called UNO CSS. And he has this article that he's written sort of about it, that I'll link in the show notes. But it's, this is essentially an engine for a tailwind, like. A framework for powering atomic CSS, but it's, it's very flexible. It's really, really easy to modify.
It doesn't come with a whole lot of like opinions or configurations out of the box. So you could build something tailwind like with it, but it's not like it's not tailwinds. Right. So if you want to have like your margins and paddings and stuff, you can add a rule to do that, but it's not, you know, sort of there out of the box.
But it's very flexible, very fast and, and sort of a really interesting abstraction. They're experimenting with using this as the engine for Windi CSS V4. So, you know, if this is something that interests you like using this mechanism for doing styling definitely check it out. Andrew if you scroll to the bottom of this, there is a really interesting mechanism that is in Windi's CSS that they're adding to this sorry.
Did, did they talk about attribute mode somewhere? I thought, yeah, attribute defy mode. It's really interesting where you can have instead of doing like a bunch of classes, you can actually do attributes in line on the element. So it looks very similar to maybe a style component. Or styled systems, I guess if you're used to style systems but it's actually still just base CSS.
Those attributes actually get put on the, the, the actual underlying elements. Anyway, there's just a lot of really interesting functionality that this, this framework or this sort of CSS build tool enables. Definitely definitely worth checking out!
Andrew: Yeah, it looks like this attributed fine mode helps kind of with your. One of the, one of the things you encounter pretty quickly with tailwind CSS is that your class name gets really, really long. And then even just figuring out what's in that class name is hard because it's a string that stretches past the, the 80 character mark by a lot usually.
this is could help.
Justin: not shown here, but they actually have it where you can just do like inline attributes. So instead of doing class equals BG blue 400, you could just do button BG, blue 400 as like a boolean attribute. And they can like target the styles for that as well. Which I thought that was like super cool.
Andrew: Oh, so this, like I was assuming for like the past minute that this would look, there's like a tool that takes the attributes and then turns them into classes. But no, this is the style sheets actually just target those attributes.
That's awesome.
Justin: That's that's exactly what it does. Yeah, it just, I mean, it's, it's all just straight CSS. It's just like changing the selectors.
Andrew: Yeah attributes are a hidden art to styling things. A great way to get around a lot of limitations and react too!
Justin: Yeah. I mean, it'll mess with your specificity and then if you're using TypeScript for the react, then you need to like, worry about like, you know, doing props and all that stuff, which is like a whole thing, not discounting the complexity there, but it seems really interesting.
Andrew: Whole different can of worms. Okay. The last tool tip of the day.
Simon: Yeah, that's something I want to check out in the future, which has been recommended to me in a Twitch channel. And I came across that tool when I was preparing slides for the last conference I spoke at and I was using reveal JS, by default, because I'm kind of used to it. But people on the, on the Twitch channel recommended that I should take a look at slide.dev.
Which is essentially us something similar to reveal JS, but it's built using Vue and Vite and also windy CSS. And it's pretty extensible and provide some pretty neat features. Like you can record your, your presentation with your camera picture and picture. So that's definitely something I want to take a look at in the near future, because.
You reveal.js is nice, but if there's something else out there which is even nicer, it's definitely worth a look!
Justin: Yeah. So slide deck for those who might have not used reveal JS slide dev as a way to use, like to write slides as a markdown. And it, you sort of like have a document and it renders it out really pretty prettily, prettily. Interesting note about this. This is also written by Anthony Fu the, the same person who did the last tool tip.
So very productive member of the coming.
Andrew: Yeah, I've said it before but we need to get him on. I think he'd be a good guest.
Justin: For sure. Anthony, if you're listening, we're coming!
Andrew: Yeah. Okay. Well, I think that about wraps it up for this week's episode of devtools.fm. Thanks for coming on. This was a fun chat about all things automation!
Simon: Once again. Thanks for having me.
Justin: Yeah, thanks again, Simon.
Andrew: Be sure to follow us on YouTube and wherever you consume your podcasts. Thanks for listening!