Esta couve flor

magic-cursor-wand: Give Your Pointer a Wand

2026-09-26T00:00:00+00:00

A small effect that follows the pointer can make a page feel a bit alive. I built one as a library and published it on npm: magic-cursor-wand.

What you get is simple to describe: chalk lines, glitter and soft clouds that follow the pointer on any web page. You can add it with one script tag, without a build step, or with one function call if your project uses a bundler.

The best way to understand it is to play with it. Open the live demo, move the pointer, hold the button down to draw, and change the sliders in the settings panel.

What it does

There are three built-in effects:

Chalk: a chalk line while you hold the pointer button down. The line fades after a short time.
Glitter: small particles that twinkle, turn and fall.
Cloud: soft clouds that rise and fade near the pointer.

All of them draw on a canvas. If you never worked with it, a canvas is an HTML element that works like a blank sheet where JavaScript can paint pixels directly. The library puts one canvas on top of the page and sets pointer-events: none on it, so your links and buttons below it continue to receive clicks as usual.

Start in one minute

With a script tag

If you have a static page, a blog or anything without a bundler, add this element at the end of the body:

<script
  src="https://cdn.jsdelivr.net/npm/magic-cursor-wand@0.2.0/dist/magic-cursor-wand.iife.js"
  integrity="sha384-XxBXqYPuFEcQp51OYVOK/CslqB4ooezSzEvohIGISSD4E7Gxi3IOg09R3e/wKmF5"
  crossorigin="anonymous"
  data-wand-storage-key="my-site-wand"
  data-wand-cursor="glow"
></script>

A few things are happening here:

The integrity attribute holds a hash of the published file. The browser compares the file it downloads with this hash and refuses to run it if they are different. Thus the browser runs only the published file.
data-wand-storage-key keeps each visitor’s settings in localStorage, so they stay after the visitor closes the page.
data-wand-cursor="glow" adds a glow around the pointer. The build adds it only on devices with a pointer that can hover, like a mouse.

With a bundler

If your project uses npm and a bundler, install the package:

npm install magic-cursor-wand

And create the wand:

import { createWand } from 'magic-cursor-wand';

const wand = createWand();

That is it. To remove the canvas and all listeners later, call wand.destroy(). There is also a React hook, useWand, in magic-cursor-wand/react, which removes the wand when React removes the component.

Make it yours

Each effect has its own settings, and each setting has safe limits. You pass the values you want to change and keep the defaults for the rest:

createWand({
  config: {
    theme: { color: '#7e23b3' },
    chalk: { size: 20, taper: 0.5 },
    glitter: { spawnRate: 6 },
  },
});

If you prefer to tune the values by eye, open the demo, move the sliders and export the result. All fields and their limits are in the configuration reference.

A few technical highlights

I tried to keep the library small and polite with the page that uses it. These are the parts I think are worth a mention:

No dependencies in the core. The canvas sleeps when nothing moves, and the frame loop also stops while the page is not visible. Each effect has a limit for its particles.
Safe on strict pages. A Content Security Policy (CSP) is a set of rules a site sends to the browser to say which scripts and styles it may run. The library sets styles with element.style only and makes no network requests, so the core and the effects need no change to the policy.
Providers for the settings. A provider is a small object that loads settings from somewhere and, if you want, saves them back. The package has providers for localStorage, an HTTP endpoint and a server event stream. After each change, the wand saves the settings automatically.
Plugins and your own effects. A plugin adds a feature without drawing on the canvas. The cursor glow and the settings panel are plugins. Effects, plugins and providers are all small objects, so you can write your own without a change to the core.
Kind to each visitor. Some people ask their operating system to reduce motion on screen. Set theme.motion to auto, and the wand obeys the prefers-reduced-motion media query: glitter and clouds turn off, and the chalk line fades without drift.

Here is how a provider looks in code. This one keeps the settings of each visitor in the browser:

import { createWand } from 'magic-cursor-wand';
import { localStorageProvider } from 'magic-cursor-wand/providers';

const wand = createWand({
  providers: [localStorageProvider({ key: 'my-site-wand' })],
});

The library works in Chrome and Edge 88, Firefox 85, Safari and Safari on iOS 14.5, and later versions. It is MIT licensed.

Try it

If you want to see it before you read anything else, the live demo is the place. The documentation has a getting-started guide, how-to pages and the full API reference. The package is on npm, and the source code is on GitHub.

If you put the wand on one of your pages, or you have an idea for a new effect, open an issue on GitHub or leave a comment below. I would be happy to see what you make with it.

magic-cursor-wand: Give Your Pointer a Wand was originally published by Cleiton Loiola at Esta couve flor on September 26, 2026.

hurl-orchestra: AI first API automation testing

2026-04-05T00:00:00+00:00

If you test an API, you probably know this flow: log in, create a cart, add an item, check out. Each step needs something from the step before it, like a token or an id. hurl-orchestra is a small tool that lets you write each of those steps as a small text file and then runs them in the right order for you. When a step breaks, it tells you which one, and it keeps running everything that does not depend on it.

Its first release on PyPI was 0.4.1. The examples below use the current version, 0.11.0, so you can copy them and run them as they are.

Why I built it

Hurl is a great tool. You write an HTTP request and its assertions in plain text, and it runs them. The trouble starts when a suite outgrows one file. You split the tests, and then you need something to call Hurl in the right order and to carry values like a login token from one file to the next. Usually that “something” ends up being a shell script, and that script only gets bigger.

I wanted to declare the order instead of scripting it. So each file now says what it needs, and the tool works out the rest.

How it looks

A Hurl file gets a small header between two --- lines (this header is called frontmatter, and it is plain YAML). Here is a login step:

---
id: auth
outputs: [token]
---
POST {{base_url}}/login
{
  "user": "ada",
  "password": "{{password}}"
}
HTTP 200
[Captures]
token: jsonpath "$.token"

A capture is a value Hurl pulls out of a response, here the token from the JSON body. The outputs list says which captures this step shares with others. Now a step that needs it:

---
id: profile
deps: [auth]
---
GET {{base_url}}/profile
Authorization: Bearer {{auth_token}}
HTTP 200
[Asserts]
jsonpath "$.user" == "ada"

deps: [auth] means “run auth first”. The token arrives as auth_token: the name of the producer, an underscore, then the output name. I did it this way so that two steps can both capture a token without stepping on each other.

Running it is two commands:

pip install hurl-orchestra
hurl-orchestra ./tests

Variables file: tests/.env
SUCCESS: auth [captured: token]
SUCCESS: profile [injected: auth_token]
Report saved to tests/report.zip

A .env file holds global values like base_url, and every Hurl call gets it. It can sit next to the tests, or anywhere else with --env-file, for example at the root of your project. The run ends with exit code 0 when everything passes and 1 when something fails, so it fits any CI.

The graph, in plain words

Behind the scenes, the files form a DAG, a directed acyclic graph. That is a fancy name for a set of steps connected by arrows (“this needs that”) where the arrows never loop back. From it, hurl-orchestra builds waves: a wave is every step whose dependencies are already done. Steps in the same wave run at the same time; the next wave starts when the current one finishes.

You can see the plan without sending a single request:

hurl-orchestra --dry-run tests/checkout.hurl

Resolved dependencies: add_item, auth, create_cart
Plan: 4 node(s)
  wave 1: auth
  wave 2: create_cart
  wave 3: add_item
  wave 4: checkout

Notice that I only asked for checkout.hurl, and it pulled in what checkout needs. The whole graph is also validated before the first request: an unknown dependency, a cycle or a field with the wrong type stops the run with a one-line error, so a broken graph never runs half a suite.

A failure stops only its branch

This is the part I like the most. When a step fails, only the steps that depend on it are skipped. Everything else keeps going, so one run shows you every broken flow, not only the first one.

In the picture, create_cart fails, so add_item and checkout are skipped, while catalog, which does not need a cart, still passes. You can play with this on the docs home page: pick a step, make it fail and watch the run.

Why “AI first”

I want to be clear here: there is no model inside hurl-orchestra, and nothing to configure. “AI first” means the files, the commands and the results are easy for a coding agent (Claude Code, Codex and friends) to write, check and read:

The tests are plain-text Hurl files with a few frontmatter fields. No code to compile.
hurl-orchestra --dry-run --json tests prints the plan as JSON, including the variable names each file can use, so an agent can check the graph before it sends any request.
--report-ctrf writes the results as CTRF, the Common Test Report Format, a JSON file with one test per request and the reason for each failure.
The whole documentation is published as a single llms-full.txt file that you can hand to an agent.

The same CTRF file also works for humans: the GitHub test reporter turns it into a test summary in your GitHub Actions job.

Under the hood

It is small on purpose. It needs Python 3.11 or later and has one dependency, python-frontmatter. The ordering comes from the standard library’s graphlib, and the waves run on a thread pool. Captured values reach Hurl through a private variables file for each call, so a token never shows up in the process list.

Where it went next

Since the first release, the project kept growing. Some of what you saw above came later:

--dry-run, and running a single file together with the steps it depends on.
The CTRF report for CI.
A retry policy with exponential backoff and jitter, for APIs that push back with a 429 or a 503.
known_failures, to tolerate a failure you already understand, with a reason and an expiry date. The report still shows it, and --strict turns it back into a failure.
The JSON plan for agents I mentioned above.
A proper documentation site, with a getting started guide, concepts and how-tos.

Try it

If you have a Hurl suite that is getting too big for one file, or a pile of shell scripts calling Hurl in order, give it a try:

And if you have a flow it cannot express yet, open an issue or leave a comment below. I would like to hear about it :)

hurl-orchestra: AI first API automation testing was originally published by Cleiton Loiola at Esta couve flor on April 05, 2026.

Understanding the Heart of the STM32: The Clock System

2025-08-09T00:00:00+00:00

The clock system is like the pulse of your microcontroller (MCU), the heart that sets the pace for everything. In the case of the STM32, which is a pretty complex chip full of peripherals, understanding how this “heart” works is essential.

Think about it with me: not everything in your project needs to run at full speed. A simple blinking LED doesn’t need the same speed as processing data over USB. Besides that, the faster the clock, the more power the chip consumes and the more sensitive it gets to interference.

The STM32 Clock Sources

The STM32 has five clock sources, which are like the “power plants” that generate the rhythm for the chip.

Note: The diagram below shows the STM32 clock tree. It shows how the clock sources (HSI, HSE, LSI, LSE) are selected, multiplied by the PLL, and distributed to the different buses and peripherals.

Looking at the diagram, we can split these sources into two categories:

Speed: High-speed sources (for performance) and low-speed sources (for saving power).
Origin: Internal sources (they already come in the chip) and external ones (you need to connect a quartz crystal).

Let’s get to know each one of them:

HSI (High Speed Internal): An 8 MHz internal oscillator. It is handy because it is already there, ready to use, but it is not the most accurate.
HSE (High Speed External): The high-speed external oscillator (usually from 4 to 16 MHz). You connect a crystal on the outside, and it gives you a clock signal much more stable and accurate than the HSI. It is the recommended one for most serious applications.
LSI (Low Speed Internal): A little 40 kHz internal “clock”. It uses little power and is meant to feed peripherals that don’t need speed, like the Watchdog (IWDG), which “watches” the system to check it hasn’t frozen.
LSE (Low Speed External): Another low-speed clock, but this one is external and very accurate, at 32.768 kHz. It is the ideal choice for the Real-Time Clock (RTC), which needs to keep the time and date right, even while the rest of the chip is “sleeping”.
PLL (Phase-Locked Loop): This guy is not an original source, but a multiplier. It takes the signal from the HSI or the HSE and multiplies it to generate a much higher frequency, reaching up to 72 MHz. It is the trick to make the STM32 run at full speed.

The Clock Path: From Start to End

The diagram we saw before shows the whole clock flow. Let’s follow the path:

Selecting the Main Clock (SYSCLK): The high-speed sources (HSI, HSE and the PLL) arrive at a selector switch. Here, the system chooses which one will be the SYSCLK, the main clock that will feed almost everything in the chip. Most of the time, we configure it to use the PLL at 72 MHz.
Distribution to the Buses: Once it is chosen, the SYSCLK is distributed. It goes through frequency dividers (prescalers) to generate slower clocks for different parts of the chip:
- HCLK: It is the clock of the AHB bus, where the CPU, the memory and the DMA live. Usually, it runs at the same speed as the SYSCLK (72 MHz).
- PCLK1: It is the clock of the APB1 bus, where the slower peripherals are (up to 36 MHz), like I2C, the USARTs and CAN.
- PCLK2: Clock of the APB2 bus, which feeds the faster peripherals (up to 72 MHz), like GPIO, SPI1 and the ADC.

Special Clocks:

USB: The USB peripheral needs an exact 48 MHz clock. This signal comes straight from the PLL, going through a special divider.
RTC: The real-time clock has its own selector switch, and it can use the LSE, the LSI or even the HSE divided by 128.
MCO (Microcontroller Clock Output): A real lifesaver for debugging! You can configure a pin (PA8) to “spit out” one of the internal clocks. That way, you can measure it with an oscilloscope and be sure everything is configured right.

Configuring Everything in Code (`SystemInit`)

All this configuration is done at the start of your code’s execution, inside a function called SystemInit. By default, STM32 projects already come with this function configured for the following scenario:

Enable the HSE (assuming an 8 MHz external crystal).
Configure the PLL to multiply the HSE by 9 (8 MHz * 9 = 72 MHz).
Select the PLL as the SYSCLK source.
Configure the dividers of the AHB, APB1 and APB2 buses.

The final result is what we see in most projects:

Clock Name	Default Speed
`SYSCLK` (System Clock)	72 MHz
`HCLK` (AHB Bus Clock)	72 MHz
`PCLK1` (APB1 Bus Clock)	36 MHz
`PCLK2` (APB2 Bus Clock)	72 MHz
`PLL` Clock	72 MHz

And the most important thing of all: no peripheral works if its clock is not enabled! Whenever you are going to use a timer, a serial port or anything else, the first thing to do in code is to turn on its “clock”.

Understanding the Heart of the STM32: The Clock System was originally published by Cleiton Loiola at Esta couve flor on August 09, 2025.

mini-painel: A Hobby Log About Talking to a Tiny USB Screen in C

2025-01-20T00:00:00+00:00

mini-painel is a hobby project of mine, and it is not finished. The idea is simple: take one of those cheap 3.5-inch USB screens and turn it into a small status panel for the desk, showing things like build status, alerts or who is on call, updating in the background with no browser tab involved.

This post is a log of where it stands. What already works, what I’m still exploring, and how you can build it and play with it yourself. The code is on GitHub, written in C99.

It started with six bytes

The very first commit was a single main.c. It opened the serial port at 115200 baud (bits per second), configured it in raw mode (no line buffering, no echo, the terminal does not touch your bytes) and sent this:

char buf[6] = {0x45, 0x45, 0x45, 0x45, 0x45, 0x45};

That is the HELLO handshake. You send six bytes of 0x45 and the display answers with six bytes of its own, which tell you which model you are talking to. Everything else was built on top of that small conversation.

Here is the part I like the most: from the computer’s point of view, this screen is not a monitor. It shows up as a serial device (something like /dev/tty.usbmodem... on macOS), the same kind of port you would use to talk to a microcontroller. If you have read my STM32 posts, this will feel familiar: read the protocol, pack bits in the right order, and respect the timing of the device.

What works today

At this point the repo can:

Find and open the screen on its own, by scanning /dev/tty.usbmodem* and picking the first device that answers HELLO.
Control the display: brightness and orientation. The XuanFang code path also knows how to set the RGB LED on the back of the Flagship model.
Draw a dashboard out of rows of components, using Cairo (a 2D drawing library) to render them. There are components for deploys, CI status, alerts, outages, on-call, sprint progress, PR review queue, sparklines (tiny line charts without axes), error rates and SLA gauges.
Split content into pages when it does not fit on one screen, and cycle through them.
Run tests without hardware, using a fake serial backend. CI builds and runs them on every push, including a pass with AddressSanitizer and UBSan, compiler tools that catch memory and undefined-behavior bugs at runtime.

A few details from under the hood

Some things that were fun to get right, all written down in the repo’s doc/ folder:

Pixel format. Your code draws in RGB888, 8 bits per color channel, 3 bytes per pixel. The screen wants RGB565, which squeezes a pixel into 16 bits (5 bits red, 6 green, 5 blue). The library converts before sending.
Packed commands. A command is a 6-byte frame where the x/y coordinates are packed across byte boundaries, with the command byte last. Bit shifting, the classic embedded pastime.
Chunks and pauses. Bitmaps go out in chunks of 2560 bytes (320 × 8 rows), which avoids USB buffer stalls on macOS, plus a 50 ms pause after each bitmap to avoid corrupted frames.
Inverted brightness. On the protocol the current build uses, 0 means brightest and 255 means darkest. The public API hides that and takes a plain percentage from 0 to 100.

Building it

You need CMake 3.10 or newer, pkg-config, Cairo and a C99 compiler. The serial code uses POSIX termios and the README lists macOS as the target. On a Mac:

brew install cairo
git clone https://github.com/klaygomes/mini-painel
cd mini-painel
mkdir build && cd build
cmake ..
make

To run the tests, which also write one image per component into bin/ so you can look at them:

cmake .. -DBUILD_TESTING=ON
make
ctest --output-on-failure

Using the library

The API is small: create a dashboard, add rows of components, render a page and push the frame to the device. This is the main loop from examples/demo.c:

dev = panel_open_auto();
if (!dev)
    fprintf(stderr, "No device found; running in headless mode.\n");
else {
    panel_set_orientation(dev, XF_ORIENT_LANDSCAPE);
    panel_set_brightness(dev, 80);
}

page = 0;
while (1) {
    const uint8_t *frame = NULL;

    if (dashboard_render_page(dash, page, &frame) < 0) {
        fprintf(stderr, "dashboard_render_page failed\n");
        break;
    }

    xf_bitmap_t bm = {.x = 0, .y = 0, .width = DISPLAY_W, .height = DISPLAY_H, .rgb888 = frame};
    if (dev && panel_display_bitmap(dev, bm) < 0)
        fprintf(stderr, "panel_display_bitmap failed\n");

    printf("Page %d / %d\n", page + 1, page_count);
    sleep(PAGE_INTERVAL_S);
    page = (page + 1) % page_count;
}

Notice that the demo keeps running without a screen, which is handy for working on the layout before plugging anything in.

A component is a plain struct with a render() callback that fills a pixel buffer, and an optional fetch() callback that refreshes its data before each frame. Three macros create them, depending on whether the component is static, reads data you own, or fetches live data:

xf_component_t logo  = XF_COMPONENT(render_logo);
xf_component_t label = XF_COMPONENT_DATA(render_text, &my_text);
xf_component_t clock = XF_COMPONENT_LIVE(fetch_time, render_clock, &time_ctx);

What I’m still exploring

This is where the “work in progress” part really shows:

Which protocol, which screen. The README talks about the XuanFang 3.5” display (Rev B or Flagship), and the repo has code for that protocol. But the project notes are written against a display that identifies as a UsbMonitor 3.5” and speaks what they call the Turing Rev A protocol, and that is what the current build uses. Both implementations live side by side for now.
Driving it without C. There is a JSON command API, where you send an array of commands like {"op":"row.add", ...} and get a JSON reply back, plus a small WebSocket daemon that accepts those commands and pushes each rendered frame to the screen.
Feeding it real data. The playground/ folder has two experiments in Python: one polls GitHub through the gh CLI for PR and build data, the other shows Claude Code session state using hook scripts. The folder name says it all: these are experiments, not features.

Some of the docs also lag behind the code (the getting-started page still shows an older panel_display_bitmap signature), which is a good sign that things are still moving.

Try it

If you have one of these screens in a drawer and enjoy this kind of tinkering, give mini-painel a try. Just keep in mind it is a hobby build and rough around the edges. If you get it running on a different model, or have an idea, open an issue or leave a comment below. I would love to hear about it.

mini-painel: A Hobby Log About Talking to a Tiny USB Screen in C was originally published by Cleiton Loiola at Esta couve flor on January 20, 2025.

Free tools for embedded development

2021-10-12T00:00:00+00:00

Below is the step-by-step you need to follow to download the tools required for development on Cortex-M controllers. In this article I mention some information specific to STM, but the guide works for any vendor.

Arm Embedded Tool Chain

It contains the set of tools needed to validate, test and compile programs developed for 32-bit Cortex processors (the Cortex-M, Cortex-R and A models).

GNU/Linux

Download the latest version available on the official site. Preferably, save the file in the /usr/share directory.

Once that is done, extract the files and delete the compressed file:

sudo tar xjf gcc-arm-none-eabi-your-version.bz2 -C /usr/share/
rm gcc-arm-none-eabi-your-version.bz2

After that, you should create symbolic links so these programs are available across the whole system:

sudo ln -s /usr/share/gcc-arm-none-eabi-your-version/bin/* /usr/bin/

You may also need to install some additional tools, like libncurses and libtinfo.

arm-none-eabi-gcc --version
arm-none-eabi-g++ --version
arm-none-eabi-gdb --version
arm-none-eabi-size --version

To check the installation, open a new terminal window and type:

arm-none-eabi-gcc --version
arm-none-eabi-g++ --version
arm-none-eabi-gdb --version
arm-none-eabi-size --version

Windows

Download the installer from the official site. Then run it and follow the Wizard’s instructions.

To check the installation, open a CMD window (Windows key + R, type CMD) and type:

arm-none-eabi-gcc --version
arm-none-eabi-g++ --version
arm-none-eabi-gdb --version
arm-none-eabi-size --version

Mac

The easiest way to install this and the other tools is using Brew; if you don’t have it yet, you can install it by running the command below:

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

Once it is installed, you only need to run the command below:

brew install --cask gcc-arm-embedded

To check the installation, open a new terminal window and type:

arm-none-eabi-gcc --version
arm-none-eabi-g++ --version
arm-none-eabi-gdb --version
arm-none-eabi-size --version

OpenOCD

It is the tool that handles the communication between the computer and the controller; it is used mainly to help with debugging, but it can also be used as a flasher. It works as a GDB server, which receives the requests made by GDB clients, and also as a telnet server.

GNU/Linux

For the most recent versions of OpenOCD, you will need libftdi (even if you have an ST-Link connector).

The easiest way to install it is using your package manager.

On Debian, for example, you need to run:

sudo apt install openocd

Check which package manager you have; it probably won’t be much different from this.

To check that it was installed correctly, type:

openocd --version

The output should be something like:

Open On-Chip Debugger 0.11.0
Licensed under GNU GPL v2
For bug reports, read
	http://openocd.org/doc/doxygen/bugs.html

Windows

The easiest way is through chocolatey, a package manager made specifically for Windows. Download choco through that link, install it, and then run this command in the command prompt:

choco install openocd

You can also download the latest version straight from the repository through this link. The file is at the end of the page and its name will be something like openocd-v0.x.x-i686-w64-mingw32.tar.gz, where x.x is the number of the latest version available.

To check that it was installed correctly, open the prompt and type:

openocd --version

The output should be something like:

Open On-Chip Debugger 0.11.0
Licensed under GNU GPL v2
For bug reports, read
	http://openocd.org/doc/doxygen/bugs.html

Mac

On the Mac, as always, the easiest way to install it is using Brew, with the command:

brew install open-ocd

To check that it was installed correctly, type:

openocd --version

The output should be something like:

Open On-Chip Debugger 0.11.0
Licensed under GNU GPL v2
For bug reports, read
	http://openocd.org/doc/doxygen/bugs.html

GNU Make

A tool used to help run repetitive tasks; it also figures out which parts of the program need to be recompiled and runs the related tasks automatically.

Linux and Mac

It already comes installed by default on these systems.

Windows

The easiest way is through chocolatey, a package manager made specifically for Windows. Download choco through that link, install it, and then run this command in the command prompt:

choco install make

You can also install it directly through this link, but pay attention: unlike choco, you will have to install the dependencies and do the configuration by hand.

CMSIS

The best way to get the most recent version is straight from the GitHub repository, at https://github.com/STMicroelectronics/STM32CubeF1/tags. You can download the files by clicking the “zip” link, right next to the date of the latest release. If you want, you can also clone the latest version straight from the project’s master or main branch.

If you are following our course on YouTube, run the code below in the root directory of your project:

git clone --depth 1 ssh@github.com:STMicroelectronics/STM32CubeF1.git stm32 && rm -rf $_/.git

It downloads the most recent files into a directory called stm32, wherever you run it.

Conclusion

With just the tools above, you will be able to develop for any processor in the Cortex-M, A and R lines, on every platform and operating system. And as you may have noticed, they have evolved a lot, and installing them gets easier every time. Now you just need to download them and start programming.

Free tools for embedded development was originally published by Cleiton Loiola at Esta couve flor on October 12, 2021.

Using GNU Make as your Dotfiles manager

2021-06-20T00:00:00+00:00

For the last years, I got used to set everything on my computer by hand. Yes, I know it is very inefficient. But, I barely have to format my laptop. I can count on one hand how many times I have formatted my computer in the last 4 years.

The motivation

In Brazil, as I was working remotely for the last couple of years, I got used to using my computer to do my job and personal stuff. So, everything that I needed was there. It lasted until I had to move, and now as I’m working in a Bank, things have changed. I saw myself being obligated to use the company computer and this computer is really restricted. To make things worse, I had to format more than one time since I started working (The first time was my fault as I messed with the configuration. The second time, an Apple update and compatibility issue made my computer unusable).

This situation taught me that I needed a dotfile repository where I must manage my configuration files. Well, the first thing I did was look at which options we had as working solutions. The most know one was GNU Stow. Well, I may say it really shines when you have a very complex configuration tree with many files. It didn’t appear to make sense for my case, where I had just a few configurations.

I wanted something that I could use with a new installation of a Mac OS and with only one command, install everything I need and all the boring stuff, and for using stow, I had to at least install it as it does not come in Mac by default.

The GNU Make for rescue

I was procrastinating on studying Make for the last years, so I thought it could be good to learn Make and do something relevant. And it was a perfect choice; before I started messing with my configuration files, I downloaded the GNU Make documentation (a 230 pages PDF), and I started reading it. I may say I got amazed at how it is straightforward, sometimes, of course, the documentation gets really dense (try to read the explanation for autovars, for example), but if you read with care, you will start getting the point of how it was projected.

In the end, I may say I’m thrilled with the result, and I would like to show what I have learned in this process.

The Makefile

Bellow is my actual make file; Basically, it manages the creation and copy of configuration files to the ~/.config folder as they are changed on the repository folder, keep them in sync. I will try to explain line by line what I’m doing and how you may change it to fit your needs.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
CONFIG		:=	$(HOME)/.config/

VIM 		:=	$(addprefix ${CONFIG}, $(wildcard nvim/* nvim/**/*))
ZSH		:=	$(addprefix ${CONFIG}, $(wildcard zsh/*))
ZSH_CONFIG	:=	${HOME}/.zshrc
GIT		:=	${HOME}/.gitconfig
BREW		:=	$(HOME)/Brewfile
BREW_ENV	:=	$(HOME)/.brewenv
MAC		:=	$(CONFIG)mac/install.sh
NODE		:=	$(CONFIG)node/globals

CREATE_TARGET_DIR=	if [ ! -d "$(@D)" ]; then echo "Directory $(@D) was not found, creating..." && mkdir -p "$(@D)";fi;

.PHONY: all mac brew git vim

all:| mac brew zsh git node vim;

.SECONDEXPANSION:
$(VIM): $$(subst ${CONFIG},, $$@)
	@$(call CREATE_TARGET_DIR)
	@if [ ! -d $< ]; then 					 \
		echo "Including ${^} configuration to ${@}" 	;\
 		cp $< $@					;\
	fi							 

.SECONDEXPANSION:
$(ZSH): $$(subst ${CONFIG},, $$@)
	@$(call CREATE_TARGET_DIR)
	@echo "Including ${^} configuration to ${@}"
	@cp "${^}" "${@}"
	@if [ "${@F}" = "setup.sh" ] ; then 	 \
		${@} "${HOME}" "${CONFIG}" 	;\
	else 					 \
		file="source '${CONFIG}${^}';" &&\
		(                                \
			grep "$${file}" ${ZSH_CONFIG} -q || echo "$${file}" >> ${ZSH_CONFIG} \
		) ;                              \
	fi

$(MAC): mac/install.sh
	@$(call CREATE_TARGET_DIR)
	@cp ${^} ${@}
	@${@}

$(GIT): $(wildcard git/*) 
	@./git/install.sh ./git/.gitconfig

$(BREW): $(wildcard brew/*)
	@./brew/install.sh
	@cp ./brew/Brewfile ${HOME}/Brewfile
	@source ${BREW_ENV} 								&&\
	($${HOMEBREW_PREFIX}/bin/brew bundle --file ${HOME}/Brewfile --force || exit 0) &&\
	./brew/setup.sh

$(NODE): node/globals
	@xargs -I {} -n 4 npm install {} --global < "${^}"
	@mkdir -p "${@}"
	@cp "${^}" "${@}"

vim:  $(VIM)
git:  $(GIT)
brew: $(BREW)
mac:  $(MAC)
node: $(NODE)
zsh:  $(ZSH)

node/globals: ;
mac/install: ;

Lines 1 to 10 are variables that hold the list of files.

The syntax $(something) is used as the same as ${something}, which means you want to interpolate the current value of something, into its current value.

But $(something ${other}...), or $(something $(other)...) tells make you are calling the function something passing the value of the variable other as its first argument (you may give more than one just separating them with spaces).

The attribution symbol := tells make I want these values to be assessed only one time while the file is being parsed; otherwise, if you use simply =, it means this value will be evaluated every time you reference it.

Note how it is desirable as I use these variables all over the place. I’m calling the special functions addprefix and wildcard and doing expression expansions (getting everything recursively inside the directories using ** ).

addprefix as its name suggests, it adds a prefix for each item of a list.
wildcard returns a list of files fetched by the expressions expansions. This function accepts more than one argument.

In short, I’m getting all files from these directories and adding ${CONFIG}, which in turns points to ${HOME}/.config/ (usually ~/.config).

However, the variable CREATE_TARGET_DIR, on line 12, is used as a function I use = as it must be evaluated every time it is called.

On line 14, I used a directive called .PHONY, which informs make which recipes its output should not be considered a valid file, or better, which ones do not produce an output file. On our Makefile, they are defined on lines 60-69 and are used only to tell make I want to run them when any file they depend on was changed. Note they use the variable with the list of files I got on lines 1-10.

On line 16, I created a recipe called all, which calls in sequence the recipes mac, brew, zsh, git, node, vim. Note I used :| to force make to execute in sequence.

The first recipe inside Makefile is called by default whenever make is called without arguments; in our file, it is the recipe all.

On lines 18-55 are the recipes for all the Makefile. Before I start explaining them I should give a little introduction to how they are formed.

A Makefile consists of a set of recipes. A recipe generally looks like this:

targets: prerequisites
   command
   command
   command

The targets are file names separated by spaces. Typically, there is only one per rule.
The commands are a series of steps typically used to make the target(s). These need to start with a tab character, not spaces.
The prerequisites are file names separated by spaces. These files need to exist before the commands for the target are run. These are also called dependencies. Each recipe can be called independently and represent a single file on the system. (fit not tagged as .PHONY)

Now let’s analyze the first recipe we have at lines 26-58.

.SECONDEXPANSION is a directive that tells Make to expand the variables contained into the following recipe more than once. You may ask why?

On line 19; first, I declare the target from the variable $(VIM), which we know is a list of files contained inside the ${CONFIG}/vim folder; these values are used as input parameters to the dependencies PLUS (it is essential) the CURRENT target which trigged the rule.

Translation: It creates a unique recipe for each target using only one dependency, that is, the target itself without the ${CONFIG} prefix.

On line 20, I call the function CREATE_TARGET_DIR, responsible for creating the target directory if it does not exist.

Note the variable $(@D); it is an especial auto variable that expands to the directory of the current target.

Note: You can declare variables as $(a), ${b} or $c inside Makefiles.

Now the variables:

$^ expands to the first dependency of the current target (our case, it is the same file without the ${CONFIG});
$@ extends to the target which fired the recipe;
${@F} same as before, but only its filename;

I must mention that:

I use @ before each recipe command; it tells Make to not print to the stdout the bash command before executing it.
Inside bash scripts, if you want to use bash variables, you must include an extra $, so it will be escaped and not interpreted as a Make variable itself.

See, it is possible to use the result of functions/variables inside targets and dependencies as actual values, so I use it a lot throughout the entire file.

Conclusion

It was an easy task, and I’m thrilled with the results as I had to learn a lot to make it possible. In the end, it was a clean solution that it will last a good time inside my workflow.

You may also be interested in the final result in my public dotfiles repository

Using GNU Make as your Dotfiles manager was originally published by Cleiton Loiola at Esta couve flor on June 20, 2021.

MiniReact: a tiny React-like library to see what happens under the hood

2018-08-23T00:00:00+00:00

If you use React every day, you probably call setState without thinking about what happens next. I had to think about it, because I wrote a small version of it as a take-home exercise for a job interview. It is called MiniReact, and in this post I will show you what it does, how it is organized and where it is intentionally simpler than React.

I published it to npm in August 2018, and you can find the whole code on GitHub.

What you get

MiniReact is not a complete React implementation, and it does not try to be. I wrote it with four ideas in mind: immutability, separation of concerns, unit tests and error handling. The README states the goal clearly: let you create trees of plain HTML elements and components, and add or remove attributes and event listeners on the fly.

In practice, it gives you:

Component, a base class for class components with props, state and setState;
node, a factory that builds virtual DOM nodes from plain objects;
MiniReact.render, which mounts everything inside a real DOM element.

When you build it, webpack outputs public/mini-react.js with libraryTarget: 'window', so these three names become globals in the browser. No JSX and no build step on your side.

Two words before we start

A virtual DOM is just a tree of plain JavaScript objects that describes what the page should look like. Creating objects is cheap; touching the real DOM is not.

Reconciliation (or diffing and patching) is the process of comparing the old tree with the new one and changing only what is different in the real DOM.

MiniReact does both, in a very small way.

Writing a component

Here is the demo app that ships with the repo (public/App.js), trimmed a bit. It is a small limit adjuster: a text input and a range slider that stay in sync.

class App extends Component {
  constructor(props) {
    super(props);

    this.state = {
      definedLimit: 2500,
      maxLimit: 5000,
    };
  }

  setDefinedLimit(e) {
    this.setState(() => ({
      definedLimit: parseInt(e.target.value)
    }))
  }

  render() {
    return node({
      tagName: 'div',
      children: [
        { tagName: 'h1', textContent: 'Ajuste de limite' },
        {
          tagName: 'input',
          type: 'range',
          min: 0,
          max: this.state.maxLimit,
          value: this.state.definedLimit,
          onchange: e => this.setDefinedLimit(e)
        }
      ],
    });
  }
}

And mounting it is one line in public/index.html:

MiniReact.render(new App(), root);

Notice the “blueprint” objects: tagName is the HTML tag, children is a list of more blueprints (or plain strings), and every other key becomes an attribute. A key starting with on becomes an event listener.

Two differences from React

setState only accepts a function. You pass a function that receives the previous state and props and returns an object. That object is deep-merged into the current state:

setState (func) {
  !isFunction(func) && throwError('state must be a function!')
  const previousState = shallowClone(this.state)
  const props = shallowClone(this.props)
  this.state = mergeObjects(this.state, func(previousState, props))
  this.__child = this.__patch(this.__child, this.render(props, this.state))
}

It is synchronous. As you can see in the last line, the component re-renders and patches the DOM right there, inside the setState call. React batches updates; MiniReact does not.

How it works: three phases

The code is split into src/vdom (the virtual DOM) and src/dom (everything that touches the real DOM). I followed a naming convention: modules whose name ends in -ing (constructing, patching, unmounting, rendering) have side effects. Everything else is meant to be pure.

Constructing

For each virtual node, MiniReact checks whether it is an element or a component. An element becomes a real DOM element with document.createElement, and its children are constructed recursively. A component gets instantiated, its render is called, and the result goes through the same process again. This is also where the component receives its __patch function, so setState can update it later.

Patching

This is the reconciliation part, and it is intentionally simple. Given the previous and next virtual node:

if the tag changed (a div became a span), the old node is thrown away and a new one is built;
otherwise, attributes are diffed. Removed attributes come out as undefined, which removes them from the DOM:

export const diff = (previousVNode, nextVNode) => Object.keys(previousVNode.attributes).reduce(
  (acc, attr) => ({[attr]: undefined, ...acc}),
  nextVNode.attributes
)

if the number of children is the same, each child is patched by position;
if it is different, the children are rebuilt from scratch.

There are no keys like in React, so reordering a list means rebuilding it. For a learning project, I think it is a fair trade.

Unmounting

The cleanup phase: walk the old tree, remove DOM nodes from their parents and drop the references so the garbage collector can do its job.

Events without leaking listeners

One detail I like: MiniReact does not attach your handler directly. Each element gets a single proxy function, onEvent, per event type, and the real handler is stored on the element itself:

function onEvent (ev) {
  this[NODE_ATTR_PROPERTY][ev.type](ev)
}

When you re-render with a new arrow function, only the stored handler changes; the listener stays the same. When the handler becomes falsy, the listener is removed. There is a test for exactly that (“should remove event listener”).

Tests

Unit tests were one of the goals from the start. There are 47 Jest test cases across four files, covering the utilities, the virtual node factory, the Component class and the DOM rendering and patching (using jsdom). The patching tests are the fun ones: they click an element and check that a div flips to a span, or that a child component is swapped for another.

Running it

git clone https://github.com/klaygomes/mini-react
cd mini-react
npm install
npm run dev   # opens http://localhost:3000
npm test

The README asks for Node >= 8.11.3 and npm ~5.6.0.

Conclusion

What started as an interview exercise made React’s behaviour feel less like magic: a tree of objects, a function that compares two trees, and a few careful DOM calls. If you are curious about how these libraries work, reading a small one is a good first step, and this one fits in a few files.

The code is on GitHub. Feel free to open it, break it, and tell me what you would do differently.

MiniReact: a tiny React-like library to see what happens under the hood was originally published by Cleiton Loiola at Esta couve flor on August 23, 2018.

The small gotcha that made me feel I know something

2018-02-01T00:00:00+00:00

Did you ever remember when you realized that [ is a built-in command in bash, like echo, cd?

Well, we as humans tend to assume things, so I did for a long time. As I came from Windows, and my programming experience is with high-level languages, in the beginning, I just got the necessary to be able to do my job as a frontend engineer.

So I learned how to create branch decisions using if, which is a shell keyword. I connected it with what I had learned in the past, associating if [ commands ]; as syntax constraint like the other languages and never worried about what I was doing; after all, I had my work done at the end.

As time went by, I realized that I had to learn more about this language; well, you know, it is impossible to work with Linux effectively without knowing it.

So I decided to start from scratch, assuming I know nothing about bash, it was a great decision, because with my mind open I started watching and reading basic tutorials; as a side note: it is sad how there is bad content on the internet, you have to take much care when you decide to learn new things because the chances you got misleading content(or just wrong) is high.

Well, I was in luck when I found the Bash Guide, written by LHunath and GreyCat. They do a great job explaining what you need to know effortlessly.

They were the first content where they take a step to make it clear that it is a command and it needs the parameter ] as the last one.

See what they say:

if executes the command [ (remember, you don’t need an if to run the [ command!) with the arguments a, =, b and ]. [ uses these arguments to determine what must be checked. In this case, it checks whether the string a (the first argument) is equal (the second argument) to the string b (the third argument), and if this is the case, it will exit successfully. However, since the string “a” is not equal to the string “b”, [ will not exit successfully (its exit code will be 1). if sees that [ terminated unsuccessfully and executes the code in the else block. The last argument, “]”, means nothing to [, but it is required. See what happens when you omit it.

(credits http://mywiki.wooledge.org/BashGuide/TestsAndConditionals)

Until that moment, everything that I thought weird about bash started making sense. Before that, I never realized why I couldn’t write things like if[$1 -lt 10] (note I didn’t use spaces) or execute commands inside the brackets like if ["$1" -eq $(cat file) ].

The simple fact that I know that [ is a command, like any normal bash program, and it expects parameters to be passed, made me never make this kind of mistake again.

I was so surprised by this little thing that I said to me even if I write anything to my blog again; it should be about it. Well, years later, here am I :)

And you when you ever realized that [ is a command?

The small gotcha that made me feel I know something was originally published by Cleiton Loiola at Esta couve flor on February 01, 2018.

GeoLocation: a take-home challenge built with React, Redux and Sagas

2017-12-22T00:00:00+00:00

If you ever had to solve a front-end take-home challenge, this post is for you. I will show how I organized one: the stack I picked, how the data flows, and a couple of small decisions I left written in the code so the reviewer (and now you) could understand why things are the way they are.

The source code is on GitHub.

The challenge

This was not a product. I built it for a job interview, as a take-home challenge. The repository started from a template prepared by Avenue Code Recruitment (its first commits, from 2014 to 2017, are theirs), and my own work went from December 18 to December 22, 2017.

The story phrase in the template was simple:

As a curious web surfer, I want to be able to locate both me and any website on a map.

In practice, the app had to:

show my own location on a map when I click “My location”, and clear it with “Reset location”;
accept a domain like nytimes.com or www.g1.com.br, call http://freegeoip.net/json/:host and show where that website is hosted;
work on desktop, tablet and phone;
have at least 80% of the code covered by tests, meaningful commits and a descriptive README.

That last item matters. The evaluation looked at the git history too, so I followed the AngularJS commit convention (feat:, fix:, test:, chore:…) and ended with 78 small commits instead of one big “done”.

The stack

React for the components, styled with styled-components (CSS written inside JavaScript).
Redux to keep the application state in one place.
Redux Saga for side effects. A “side effect” here is anything that talks to the outside world, like an HTTP request.
Next.js for server-side rendering (SSR), which means the first HTML is produced by the server instead of the browser.
Jest and Enzyme for tests.
Storybook, a small catalog where each component can be viewed and played with in isolation.

How the data flows

Every component is “dumb”: it receives data through props and reports user actions through callbacks. A higher-order component (a function that takes a component and returns a new one) called WithRedux connects them to the store:

dispatch => ({
  requestHostnameLocation (hostname) {
    dispatch(actionsCreators.hostnameLocationFetch(hostname))
  },
  requestMyLocation () {
    dispatch(actionsCreators.myLocationFetch())
  },
  resetMyLocation () {
    dispatch(actionsCreators.myLocationReset())
  },
  // ...
})

When a FETCH_* action is dispatched, a saga picks it up. Both “my location” and “website location” hit the same API, so I used one generator and a small map to decide which action to fire on success or failure:

export const fetchLocation = function * fetchLocation ({type, payload}) {
  const actions = actionMapper[type]

  try {
    const {data} = yield call(axios.get, `${API_URL}${payload || ''}`)
    yield put(actions.store(data))
  } catch (err) {
    yield put(actions.fail(err))
  }
}

With no payload the URL is just the API root, which returns the caller’s own location. With a hostname it returns the website’s location.

The root saga uses takeLatest, so if you click “Locate” twice quickly only the last request wins. Errors got a single place too: one saga watches for any action whose type starts with FAIL and turns it into a message on screen.

export const watchForErrors = function * watchForErrors () {
  while (1) {
    const {error} = yield take(({type}) => /^FAIL/.test(type))
    yield put(actionsCreators.errorShow(error))
  }
}

This way, a new request type gets error handling for free, as long as its failure action follows the naming pattern.

Validating the input

The form had to accept only domains, with or without www.. The validation is a single regular expression in utils/is-valid-domain.js:

const REG_URL = /^[a-zA-Z0-9\-.]+\.[a-zA-Z]{2,3}(:[a-zA-Z0-9]*)?$/

export default (url) => REG_URL.test(url)

If it fails, the <HostnameDispatcher /> component selects the text and shows “Please inform a valid hostname” instead of sending the request.

A design decision I left in the code

My last commit was only a comment, and I think it is the most interesting one. In <LocationTable />, each row has an info icon with its own click handler. The easy way would be to write onClick={this.onRequestInformation.bind(this, 'city')} in render. The problem is that bind creates a new function on every render, so a component that only re-renders when its props change (a “pure” component) always sees a “new” prop and re-renders anyway.

So I created all the handlers once, in the constructor:

// I know it should be easier/readeable to use this.onRequestInformatino.bind(this, param1, param2...)
// for each onClick, but it brings some performance implications.
// more info: https://medium.com/@esamatti/react-js-pure-render-performance-anti-pattern-fb88c101332f
this.handleOnRequestInformation = Object.keys(this.descriptionTable).reduce((acc, cur) => {
  acc[cur] = () => {
    if (typeof this.props.onRequestInformation === 'function' && this.props.lastUpdate) {
      this.props.onRequestInformation(cur, this.descriptionTable[cur], this.props.lastUpdate)
    }
  }
  return acc
}, {})

As the commit message says: pure render optimized components can be very fast, but they require you to treat data as immutable, and in JavaScript that can be challenging sometimes.

What I left out

The solution page listed what was missing, and I prefer to be upfront about it: fixing FOIT (the flash of invisible text while web fonts load), caching API calls, more tests, webpack optimization and i18n.

Update, September 2026: running it today

The README points to a Heroku demo, a hosted Storybook and a coverage report. Those links no longer work, and freegeoip.net itself no longer answers, so the demo is not online anymore. The code is still there: nine components have stories you can open with npm run storybook, and npm run test:coverage generates the coverage report locally. Keep in mind the README asks for Node 8.

Conclusion

It was a small app, but a good excuse to put Redux Saga, SSR and Storybook together in a few days and to practice writing commits someone else would read.

If you are preparing for a challenge like this, feel free to browse the repository and the commit history. And if you would have organized it differently, tell me in the comments.

GeoLocation: a take-home challenge built with React, Redux and Sagas was originally published by Cleiton Loiola at Esta couve flor on December 22, 2017.

Introduction to async and await (in Portuguese)

2016-06-26T00:00:00+00:00

English readers: this is my authorized Portuguese translation and adaptation of “Async and Await” by Stephen Cleary. The original is in English, so go read it there.

Este artigo é uma tradução e adaptação autorizada do original “Async and Await”, de autoria do MVP Stephen Cleary.

Introduzindo as palavras-chaves

Irei usar alguns conceitos que serão explicados posteriormente, por isso, acalme-se e concentre-se nesta primeira parte.

Métodos assíncronos tem a seguinte aparência:

1
2
3
4
5
6
public async Task DoSomethingAsync()
{
  // No mundo real, nós enventualmente fariamos alguma coisa…
  // Para este exemplo, vou apenas esperar de forma assícrona por 100ms.
  await Task.Delay(100);
}

A palavra-chave async habilita a palavra-chave await naquele método e modifica como o resultado é tratado. E isso é tudo que async faz! Não faz com que ele rode em um thread pool ou algum tipo de magia negra, a palavra-chave async apenas habilita a palavra-chave await(e gerencia o retorno dos métodos).

O início de um método async é executado como qualquer outro, ou seja, roda de forma síncrona até que ele alcance o primeiro await (ou lance uma exceção).

A palavra-chave await é onde as coisas se tornam assíncronas. await é como um operador unário: Ele recebe um argumento simples, um awaitable (uma operação assíncrona) e o examina para ver se ele já completou; se ele estiver completado, o método apenas continua(de forma síncrona) como um método normal, mas se o await notar que o awaitable ainda não completou, ele atua de forma assíncrona, pedindo para o awaitable rodar o restante do método até que se complete e devolve à execução para quem chamou o método async.

Mais tarde, quando o awaitable completar, ele executará o restante do método async. Se você usar a palavra-chave await em um awaitable padrão(tal como Task), o restante do método async irá executar no contexto que foi capturado antes do await ter sido retornado.

Gosto de pensar no await como asynchronous wait(espera assíncrona). Isso quer dizer que o método async pausa até que o awaitable complete (então ele espera), mas a thread atual não é bloqueada (por isso ele é assíncrono);

Awaitables

Como eu mencionei await recebe um argumento – um awaitable – que é uma operação assíncrona. Existem dois tipos disponíveis de awaitables no Framework .NET: Task<T> e Task.

Há também outros awaitable: métodos especiais tais como Task.Yield que retornam awaitables que não são Tasks e o Runtime WinRT que possui tem um awaitable não gerenciado. Você também pode criar o seu próprio (geralmente por razões de performance) ou usar métodos de extensão para fazer um tipo que não é awaitable se tornar um. Isso é tudo que direi sobre como criar os seus próprios awaitables, já que em todo o tempo que eu usei async/await poucas vezes tive que escrever um para mim. Se você quiser saber mais sobre como escrever um personalizado, veja o blog do time de parallel e o blog do Jon Skeet.

Um ponto importante sobre os awaitable é: O tipo que é awaitable e não o método. Em outras palavras, você pode usar o await em um resultado de um método async que retorna uma Task… Porque o método retorna uma Task, não por que ele é async. Logo, você também pode usar o await no resultado de um método não assíncrono que retorna uma Task.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
public async Task NewStuffAsync()
{
  // Use await e se divirta com a coisa nova
  await ...
}

public Task MyOldTaskParallelLibraryCode()
{
  // Note que este método não é async então não podemos usar await aqui
  ...
}

public async Task ComposeAsync()
{
  // Nós podemos chamar o await em Tasks, não importando de onde elas vieram.      await NewStuffAsync();
  await MyOldTaskParallelLibraryCode();
}

Dica: Se você tem um método assíncrono muito simples, você pode escreve-lo sem usar a palavra-chave await (ex. usando Task.FromResult). Se você puder escrever sem, faça e também remova a palavra chave async. Um método não async que retorna um Task.FromResult é mais eficiente que um método async que retorna um valor.

Tipos de retorno

Métodos async podem retornar Task<T>, Task e void. Na maioria dos casos, você deverá retornar um Task<T> ou Task e void apenas quando realmente for necessário.

Por que retornar Task<T> ou Task? Por que eles são awaitables e void não. Se você tem um método assíncrono que retorna uma Task, você poderá passar o resultado para um await, já com o void, você não terá nada o que passar, em resumo, você só deve usar void quando estiver escrevendo event handlers assíncronos.

Dica: Você também pode usar async void para outros tipos de ações, ex. um método único do tipo static async void MainAsync() para aplicações de console. Porém, este uso do async void tem seus próprios problemas; veja Async Console Programs. O caso de uso principal para métodos async void é para event handlers onde a conclusão/retorno do método não é importante.

Retornando valores

Métodos assíncronos retornando Task ou void não possuem um valor de retorno, já os métodos que retornam Task<T> deve retornar um valor do tipo T.

1
2
3
4
5
6
7
public async Task<int> CalculateAnswer()
{
  await Task.Delay(100); // (Probably should be longer...)

  // Retorna um "int" não uma "Task<int>"
  return 42;
}

É um pouco estranho de se acostumar, mas há boas razões por trás deste design.

Contexto

Em nossa introdução, eu mencionei que quando você usa a palavra-chave await em um tipo awaitable padrão, ele irá capturar o “contexto” e depois aplicá-lo ao restante do método, mas o que exatamente é este “Contexto”?

Uma resposta simples:

Se você está na thread de UI, então o contexto será o de UI.
Se você estiver espondendo a uma requisição ASP.NET, então ele será o contexto da requisição.
Caso contrário, geralmente será o contexto do thread pool.

Reposta complexa:

Se o SynchronizationContext.Current não for null, então ele será o atual SynchronizationContext. (Os contexto de UI e requisição do asp.net são SynchronizationContexts)
Caso contrário, será o TaskScheduler atual (TaskScheduler.Default é o contexto do thread pool). O que isso quer dizer no mundo real?Uma coisa, a captura e recuperação e dos contextos de UI/ASP.NET são feitos de forma transparente.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
// Exemplo de WinForms (Funciona do mesmo jeito para WPF).
private async void DownloadFileButton_Click(object sender, EventArgs e)
{
  // Uma vez que a gente espera assíncronamente, a thread de UI não é bloqueada pelo download do arquivo.
  await DownloadFileAsync(fileNameTextBox.Text);

  // Após a conclusão, nós retomamos no context de UI e podemos acessar facilmanete elementos de UI.
  resultTextBox.Text = "File downloaded!";
}

// Exemplo em ASP.NET
protected async void MyButton_Click(object sender, EventArgs e)
{
  // Já que a gente está esperando de forma assíncrona, a thread do ASP.NET não é bloqueada pelo download do arquivo.
  // isso permite que ela fique disponível para executar outros requests neste meio tempo.
  await DownloadFileAsync(...);

  // Depois de finalizado, retomamos o context e podemos acessar os dados do request atual.
  // Nós poderemos estar em outra thread, mas nós teremos o mesmo context ASP.NET.
  Response.Write("File downloaded!");
}

Isso é ótimo para event handlers, mas nem sempre será o que você irá precisar para maioria do código (que será boa parte código assíncrono que você escreverá).

Evitando o Contexto

Na maior parte do tempo, você não precisa voltar ao contexto principal. Boa parte dos métodos assíncronos são projetados com composição em mente: Eles usam o await de outras operações, cada um representando uma operação assíncrona por si mesma (podendo ser compostas por outras). Neste caso, você pode dizer ao awaiter para não capturar o contexto atual chamando ConfigureAwait e passando false para ele. Exemplo:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
private async Task DownloadFileAsync(string fileName)
{
  // Use HttpClient or whatever to download the file contents.
  var fileContents = await DownloadFileContentsAsync(fileName).ConfigureAwait(false);

  // Note that because of the ConfigureAwait(false), we are not on the original context here.
  // Instead, we're running on the thread pool.

  // Write the file contents out to a disk file.
  await WriteToDiskAsync(fileName, fileContents).ConfigureAwait(false);

  // The second call to ConfigureAwait(false) is not *required*, but it is Good Practice.
}

// WinForms example (it works exactly the same for WPF).
private async void DownloadFileButton_Click(object sender, EventArgs e)
{
  // Since we asynchronously wait, the UI thread is not blocked by the file download.
  await DownloadFileAsync(fileNameTextBox.Text);

  // Since we resume on the UI context, we can directly access UI elements.
  resultTextBox.Text = "File downloaded!";
}

O que é importante notar com este exemplo é cada nível de chamada de um método assíncrono tem seu próprio contexto. DownloadFileButton_Click inicia no contexto da UI, chama DownloadFileAsync. DownloadFileAsync também inicia no contexto da UI, mas então o deixa para trás ao chamar ConfigureAwait(false). O resto do método DownloadFileAsync executa no contexto da thread pool. Porém quando DownloadFileAsync completa, o DownloadFileButton_Click continua, ele, de fato, continua no contexto da UI.

Uma boa regra a seguir é de sempre usar ConfigureAwait(false) a menos que você saiba que realmente irá precisar do contexto.

Async Composition

Até agora nós apenas consideramos composição serial: um método assíncrono aguarda por uma operação a cada tempo, mas também é possível iniciar várias operações e esperar por uma (ou todas) completarem. Você pode fazer isso iniciando operações, mas não usando await imediatamente.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
public async Task DoOperationsConcurrentlyAsync()
{
  Task[] tasks = new Task[3];
  tasks[0] = DoOperation0Async();
  tasks[1] = DoOperation1Async();
  tasks[2] = DoOperation2Async();

  // Neste ponto, todas as três tarefas estão rodando ao mesmo tempo. 
  // Agora a gente await todas elas de uma vez.
  await Task.WhenAll(tasks);
}

public async Task<int> GetFirstToRespondAsync()
{
  // Chama dois web services; Pega a primeira resposta
  Task<int>[] tasks = new[] { WebService1Async(), WebService2Async() };

  // Await pelo primeiro a responder
  Task<int> firstTask = await Task.WhenAny(tasks);

  // Retorna o resultado
  return await firstTask;
}

Ao usar composição concorrente (Task.WhenAll ou Task.WhenAny), você pode executar operações concorrentes simples. Você também pode usar estes métodos junto com Task.Run para fazer computações simples em paralelo. Mas isso não é um substituto para a Task Parallel Liberay – qualquer operação que faça uso intensivo de CPU e operações paralelas devem fazer através da TPL.

Orientações

Leia o documento da Task-based asynchronous (TAP). Ele é extremamente bem escrito e inclui informações sobre o design da api e uso apropriado das palavras chaves async e await assim como inclusão de eventos de cancelamento e progresso.

Existem muitas novas técnicas compatíveis com await que devem ser usadas ao invés das técnicas antigas onde há bloqueio da aplicação. Se você tem alguns desses antigos exemplos no seu código novo com async, você está fazendo isso errado:

Antigo	Novo	Descrição
task.Wait	await task	Aguarda/await uma task completar.
Task.Result	await task	Pega o resultado de uma task completada
Task.WaitAny	await Task.WhenAny	Aguarda/await que uma de uma coleção de tasks complete.
Task.WaitAll	await Task.WhenAll	Aguarda/await que todas as tasks de uma coleção complete
Thread.Sleep	await Task.Delay	Aguarda/await por um periodo de tempo.
Task constructor	Task.Run or TaskFactory.StartNew	Cria uma task baseada em código

Próximos passos

Stephen Cleary publicou um artigo na MSDN intitulado Best pratices in Asynchronous Programming (Melhores práticas em programação assíncrona) que explica de forma mais completa as orientações sobre “evite void async”, “assíncrono por todo caminho” e “configure o contexto”. A documentação oficial da MSDN é muito boa; Eles incluem uma versão online do documento Task-based Asynchronous Pattern que é excelente e cobre os designs dos métodos assíncronos. A equipe async publicou uma faq sobre async/await, um grande lugar para continuar aprendendo sobre async. Eles possuem destaques dos melhores posts e vídeos por lá. E também qualquer blog post do Stephen Toub é bem instrutivo. E claro, outro recurso é o blog do Stephen Cleary. Todo este material em inglês.

Introduction to async and await (in Portuguese) was originally published by Cleiton Loiola at Esta couve flor on June 26, 2016.

Covariance and contravariance in C Sharp

2016-06-18T00:00:00+00:00

First, we need to define what variance is, along with its daughters, covariance and contravariance. To start, you need to understand that the type systems of most object-oriented programming languages allow new types to be created from other ones using inheritance, and that these new types can be used anywhere the base types would be expected.

Example:

If you have a class Cat that inherits from Animal, you can use it in any method that accepts only Animal as a parameter:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
public abstract class Animal
{
	public abstract void WagTail();
}
public class Cat : Animal
{
	public override void WagTail()
	{
		System.Console.WriteLine("wagging frantically");
	}
}
public static void Call(Animal animal)
{
	animal.WagTail();
}
public static void Main(string[] args)
{
	var cat = new Cat();
	Call(cat);// <<--compiles without problems
}

This behavior is called Polymorphism, and you have probably heard about it already, but what do you do if the method, instead of just one Animal, expected a list of Animal?

When we are working with type containers (in plain words: classes that only exist as a kind of box shaped exclusively to give access to specific types), these metamorphoses start being called variances, and they are classified into three subgroups: covariance, contravariance, and invariance.

Now that you know this exists, let’s dig a little deeper; look at the following example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
public abstract class Animal
{
	public abstract void WagTail();
}
public class Cat : Animal
{
	public override void WagTail()
	{
		System.Console.WriteLine("wagging frantically");
	}
}

public static void Main(string[] args)
{
	Cat[] cats = new Cat[] { new Cat(), new Cat()};
	Call(cats);// <<--cats, which is of type Cat[], is converted to Animal[], a more generic type
}

public static void Call(Animal[] animals)
{
	foreach(var animal in animals)
		animal.WagTail();
}

In the example above, note that the System.Array type, the container for the Animal and Cat types, allows its metamorphosis whenever the type it holds is a base type of the other System.Array’s type; we call this mutation covariance.

In other words, covariance happens whenever a container object initialized with a more specialized type can be assigned to a container object that holds a more basic one.

Contravariance, on the other hand, is the opposite, where only specialized types accept more basic types.

It doesn’t seem to make sense, but it does, and you have probably already used contravariance in your day-to-day work; look at this example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
public static void Log(object data)
{
	Console.WriteLine($"Called with {data} type of {data.GetType().Name}");
}

public class Writter
{
	private string _data;
	public Writter(string data)
	{
		_data = data;
	}
	public  override string ToString()
	{
		return $"Writter says '{_data}'";
	}
}

public static void Main(string[] args)
{
	Action<string> callLogAsString = Log; //<-- Log is of type Action<Object>, converted to Action<string>; note that string is more specific than object.
	Action<Writter> callLogAsWritter = Log;
	
	callLogAsString("foo");
	callLogAsWritter(new Writter("bar"));
	
}

In the example above, starting from the signature of a generic method, it was possible to convert it to a more specific type.

Now let’s see what problems these kinds of metamorphoses can get us into. In the first example, we saw that System.Array is covariant, which makes it possible for the following code to compile but fail miserably at runtime:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
public class Camel : Animal
{
	public override void WagTail()
	{
		System.Console.WriteLine("wagging gently");
	}
	public void DrinkWater()
	{
		System.Console.WriteLine("Drinking 5 liters of water");
	}
}
public static void Call(Animal[] animals)
{
	foreach(var animal in animals)
		animal.WagTail();
animals[0] = new Camel();//<--here I will get a runtime error (ArrayTypeMismatchException)
}

Note that it is safe to read properties and call methods of covariant containers, but dangerous to write. In the example above, the Call method received an Array of Cats, but tried to write a Camel, which obviously doesn’t fit, causing a runtime error.

In C#, starting from version 4, we can control the variance of our types using the in and out keywords.

This control can only be done on generic interfaces or generic delegates; it is not allowed on classes and other types. The way the types are exposed is also controlled, in this last case to guarantee the safety of the data exposed by the containers. Look: The in keyword defines types that are contravariant.

1
2
3
4
5
6
7
public interface IKlay<in T>
{
	void Receive(T something);
}

IKlay<Camel> specificKlay = new Klay<Animal>();
specificKlay.Receive(new Cat());

Although they are of different types, it is safe, since the method only knows about animals (which cat derives from), and don’t forget that the one doing the work will be Klay and not Klay. The out keyword defines covariant types, which are only safe when read from the containers. That's why it is only possible to define covariant types as method return types:

1
2
3
4
5
6
public interface IKlay<out T>
{
	T Send();
}
IKlay<Animal > genericKlay  = new Klay< Camel >();
Animal sent = genericKlay.Send();

It is totally safe to read the type sent by the covariant generic type.

Covariance and contravariance in C Sharp was originally published by Cleiton Loiola at Esta couve flor on June 18, 2016.

My new challenge: Learning F Sharp

2016-06-16T00:00:00+00:00

New challenge

In July, I decided to challenge myself to learn something new; this time it will be the F# language. To do that, there is nothing better than starting by reading material we enjoy and practicing.

The reason for this new challenge: besides strengthening my set of options as a solutions architect, I want to be able to apply these new concepts to the tools I already use.

I started reading the book “The book of F# - Breaking free with managed functional programming” by Dave Fancher. This book has only 314 pages, and I liked the approach, since the author goes straight to the point; right in the first chapter he makes that clear by presenting a simple reverse Polish notation parser. That makes it not very suitable for those who have had little contact with the programming world.

As I am still pretty green at this beginning and don’t have much to write about, I will start by translating the best content I find on the internet and publishing it here, without forgetting, of course, to give proper credit to its real authors. As I get more familiar with it, I will publish my own content about possible gotchas or random information about the language.

My new challenge: Learning F Sharp was originally published by Cleiton Loiola at Esta couve flor on June 16, 2016.

Why we are staying with Microsoft's .NET stack

2016-05-02T00:00:00+00:00

This year, we had the opportunity to take a look at, and decide whether it would be viable, moving from Microsoft technology to a Linux environment using Python. I decided to write this article as a summary of our meetings and of why I decided to keep using the Microsoft stack.

Technical motivations.

Advanced concepts about scalable applications are abstracted away from developers. In the .NET framework, we have native support for multithreading and parallelism through syntactic sugar (async and await). Programmers don’t need to know about these more advanced techniques to make use of them. In Python, on the other hand, the project’s architecture and infrastructure need to be written with this in mind from the start.

C#, being a language compiled at runtime, is by nature faster than interpreted languages. Compared with Python, it can be up to 50,000% faster.

This can make the application harder to maintain, since to work around this problem, Python allows CPU-heavy operations to be written in C++, which requires a professional who knows that language and the implications of its interoperability.

It is easier to maintain applications written in typed languages. With a strongly typed language, even though productivity while developing new features may suffer compared with dynamic languages, this time is paid back during maintenance. A good part of development costs happens during maintenance; dynamic languages require programmers to be more disciplined and to know the environment they are developing in well, so that they produce more robust and easier-to-maintain applications.

Compilation as a code quality tool. One of the advantages of interpreted languages over compiled ones is the cost of compilation during the development workflow. However, it can turn into a disadvantage, since errors like syntax errors, or calls to properties or methods that don’t exist, are only caught when the code runs, which demands more effort on test coverage and, in the end, increases the development cost.

Team maturity. Today our team has great professionals with an excellent background in the Microsoft architecture; with a migration, we would need to adapt and wait for the learning curve. Remember: your team is only as good as your worst programmer.

Cloud infrastructure

Both Google’s and Microsoft’s solutions are excellent, highly scalable, cheap, and secure. Microsoft’s solutions have a price advantage for some offerings, for example: for a micro virtual machine, the minimum price charged on Google Cloud is around 130 dollars (450 reais), while on Microsoft’s platform it is 17 dollars. There is also the location of their data centers; Google doesn’t have servers in Brazil yet. However, it is possible that commercial agreements or implementation details make up for the amounts paid. In the current market, there are no advantages or disadvantages that decide the choice of one over the other; they are all integrated with their respective development environments, and both let you publish entire applications with one click.

Result

The new project starts in July, and we are going to build it in .NET and C#. The version chosen for development will be 4.6.1, MVC 5 / WebAPI 2, using SignalR.

Right now, my team and I are planning the architecture, developing the infrastructure, and validating it on top of sketches with complex use cases. In the next articles, I will post updates with details on how it is turning out.

Why we are staying with Microsoft's .NET stack was originally published by Cleiton Loiola at Esta couve flor on May 02, 2016.

Migrating from TFVC to Git

2016-01-30T00:00:00+00:00

Almost 10 years later, we are going to start using GIT. Follow our story and see a small manual so you can use what you already know about TFVC to learn how to use GIT in Visual Studio. In this article I only talk about file versioning; the collaborative tools, checkin policy, issue tracker, etc. will have their turn as soon as we get used to GitHub.

Our little story

I remember as if it were yesterday the excitement of setting up the TFVC client; back then we were migrating from the infamous Microsoft SourceSafe to what was, at that time, the promise of better days at work.

Team Foundation Version Control was beautiful; it came with the promise of being the ideal tool for versioning files in bigger projects, robust and fully configurable, and its wonderful diff and merging, which really worked, were a success (and still are) inside our team.

We said goodbye to the \\DONEBY: annotations and to the hours lost with practically manual merges. The winds of change were whistling in our office.

Almost 10 years later, once again driven by necessity, we are migrating, this time from our beloved companion TFVC to GIT. But why? It sounds silly, but the main reason is that it is not fast enough.

Although there is no limit to the number of items you can have in the same workspace, when this number grows, Visual Studio’s performance drops drastically; that is indisputable. Of course, there are ways to reduce this effect, but the fact is that there comes a moment when nothing helps anymore, and so we are forced to look for new options.

Now GIT, besides being an excellent version control system, robust and maintained by a huge community, has as its main promise being extremely fast.

And it has been! Our initial tests showed that using it with Visual Studio will make our work countless minutes faster. A clear example is when we decide to switch from one branch to another; a process that in TFS took on average up to a full minute happens almost instantly with GIT¹.

It proves to be a natural choice for those who got used to a product that is easy to use and just works.

First steps with GIT for a TFVC Developer

Git is much simpler than TFVC, and the first thing you will notice when you start using it will be its terminology, some slightly different concepts, and missing and/or extra features, which end up making everything look confusing; but don’t worry, below I list the main differences between the two.

Goodbye workspaces

A mandatory and very important feature in TFVC does not exist in GIT. To start working with GIT you don’t need to create workspaces, configuring directories, file locations, permissions, etc. Just choose a directory on your computer, clone the repository you want to work on, and get to work.

Branches are simpler

It really shines here; unlike TFVC, which forces you to copy all the files of the current project into a different directory for a new branch, GIT is smart enough to manage by itself the differences between the files of each branch using the same directory.

No more Check ins or Check outs

Git has no concept of file lock; you can edit and ‘submit’ any file at any time to the central repository on any branch², but be careful: checkout in GIT has another meaning.

This has an interesting effect: to avoid future headaches, from the very beginning teams find themselves obliged to agree among themselves and work following a certain pattern to make sure there are no conflict problems.

Among them, the most known and used is git-flow, which basically uses branches to organize the updates to the code base.

Local and server repositories

Just like TFVC has local and server repositories, so does GIT, but they are conceptually different.

In TFVC, when you use the local repository option, on every operation (checkin, checkout, shelve, etc.) TFVC checks the content of each file in the workspace against the files in the server repository; for server repositories, the check is only on the read-only flag.

In GIT it is different: you will always work on your local repository, sending to the server only when the work is ready.

Commits, fetch, pull, push and sync

In TFVC there are basically two ways to work with files:

check out, which basically tells TFVC that you want to edit the file, and check in, where you send the changes you made to the central repository, visible to everyone.
shelve and unshelve, similar to the previous one, but the changes are available only to you.

In GIT the workflow is different; first you need to understand that your files will always be in one of four different states:

untracked: as the name says, these are files that git is not tracking. Files that were not present in the last commit and have not been staged yet;
unmodified: files that were in the last commit and were not modified;
modified: files that were in the last commit and were modified but have not been staged yet;
staged: files that are ready to be part of the next commit;

File state flow

Where from and where to a file can ‘jump’ during its lifecycle, source: https://git-scm.com/

commit

A good analogy would be to compare them with the Windows restore feature, where ‘restore points’ are saved before important changes are made to the system, allowing you, in case things go wrong, to go back to a safe version.

Commits in GIT are an important tool; besides letting developers save their progress by marking safe points they can go back to if something goes wrong, when well done they can also be an excellent way to automate documentation.

Don’t forget that only staged files are recorded by the commit.

fetch

Downloads the named HEADs or tags from the server repository, along with their objects, updating your local repository WITHOUT doing a merge.

A super simplification would be to think that the files on your computer have reference links to a commit version on the server. When this version changes, you need to update this reference before you can send new versions. What fetch does is download the updates that happened since the last fetch and save them in a ‘special area’ that you can inspect, if you want, before doing a merge.

pull

Pull, besides fetching the files like fetch does, also does the merge automatically for you. See this example.

Imagine this is the state of your repositories:

	  A---B---C master on the server
	 /
    D---E---F---G master <- local repository
	^
	origin/master last fetch (or pull)

After the PULL, git will update the references of commit G with B and C, which were made on the server, creating a new one from the merge called H.

	  A---B---C master on the server
	 /         \
    D---E---F---G---H <- local repository

push

When you have finished your work, fixed that bug or feature, use this command to send your changes to the server repository. This command will take all the changes that were committed locally and send them to the server. But be careful: you can only ‘send’ if the local/remote references are up to date, so it is always a good idea to check using fetch.

sync

Think of sync as everything said above, but done from a single user operation. It will do whatever is needed so that your local repository becomes equal to the remote one: fetching updates (fetch), merging (merge) and sending your changes (push), leaving some work for you to do only if there is a conflict.

Conclusion

Without a doubt GIT is an excellent tool that made our development cycle much more productive. Every day that goes by we learn more, and the easier its daily use gets. It really was an excellent decision that will last for a few more years.

Computer with 16GB of RAM, SSD, I7 processor and a dedicated 10MB fiber optic internet connection, using GitHub and Visual Studio Team Services. ↩
If you have write permission, of course. ↩

Migrating from TFVC to Git was originally published by Cleiton Loiola at Esta couve flor on January 30, 2016.

karma-typescript-preprocessor2: running TypeScript tests in Karma without temp files

2015-11-11T00:00:00+00:00

In 2015, if you had TypeScript code and browser tests, you needed some glue in between. This post is about the glue I wrote: karma-typescript-preprocessor2. You will see what it solved, how to configure it, and how it worked inside.

A note on timing: I’m filing this post on the day version 1.0.0 went to npm, November 11th, 2015, but I’m writing it looking back, so I will also tell you where the project ended up.

First, a few words

Test runner: a program that finds your tests, runs them and reports what passed and what failed. Karma is a test runner that runs your JavaScript tests inside real browsers.
Transpile: translate code from one language to another at the same level. TypeScript is transpiled to JavaScript, because browsers only understand JavaScript.
Preprocessor: in Karma, a plugin that transforms each file before it is sent to the browser. Here, it takes a .ts file and hands Karma the .js version.

The problem

Karma can’t run a .ts file, so something has to compile it first.

A preprocessor for that already existed on npm: karma-typescript-preprocessor, published by Sergey Todyshev in 2013, described as “a karma-runner plugin to compile typescript files on the fly”. Mine is not a fork of it; the git history starts from scratch. But the name was taken, so I added a 2 at the end. The README even says it: “note number 2 at the end”. Both plugins register the same typescript preprocessor name and read the same typescriptPreprocessor section of karma.conf.js.

What mine does differently is written in the first line of its README:

Transpile files in memory using gulp-typescript. No temp files are generated.

How to use it

Install it as a dev dependency:

$ npm install karma-typescript-preprocessor2 --save-dev

Then point Karma at it. This is the full example from the README:

// karma.conf.js
module.exports = function(config) {
  config.set({
    files: [
      '**/*.ts'   // Preprocessor will convert Typescript to Javascript
    ],
    preprocessors: {
      '**/*.ts': ['typescript', 'sourcemap']   // Use karma-sourcemap-loader
    },
    typescriptPreprocessor: {
      // options passed to typescript compiler
      tsconfigPath: './tsconfig.json', // *obligatory
      compilerOptions: { // *optional
        removeComments: false
      },
      // Options passed to gulp-sourcemaps to create sourcemaps
      sourcemapOptions: {includeContent: true, sourceRoot: '/src'},
      // ignore all files that ends with .d.ts (this files will not be served)
      ignorePath: function(path){
       return /\.d\.ts$/.test(path);
      },
      // transforming the filenames
      // you can pass more than one, they will be execute in order
      transformPath: [function(path) { // *optional
        return path.replace(/\.ts$/, '.js');
      }, function(path) {
         return path.replace(/[\/\\]test[\/\\]/i, '/'); // remove directory test and change to /
      }]
    }
  });
};

The short version of each option:

tsconfigPath is required. The plugin refuses to start without it.
compilerOptions lets you override (or add) anything from tsconfig.json.
ignorePath skips files you don’t want served, like .d.ts definition files.
transformPath renames the virtual path Karma sees; the example strips a test folder so specs can import modules from src.
sourcemapOptions is passed to gulp-sourcemaps.

The one opinion it had

The plugin only accepts configuration through tsconfig.json. In the README’s words, “a lot of problems with typos and references are completely solved, as compiler will use same basedir to resolve files”.

How it worked inside

The whole plugin is a single index.js. Three ideas carry it:

Compile the project, not the file. When Karma starts, the plugin calls ts.createProject(tsconfigPath, compilerOptions) from gulp-typescript and compiles everything once. The output goes into a writable stream that keeps each compiled file in an in-memory buffer. Nothing touches the disk. The README credits gulp-typescript’s incremental compilation for fast rebuilds.
A tiny state machine. The plugin is idle, compiling or compilationCompleted. If Karma asks for a file while a compilation is running, the request waits in a queue. When compilation ends, the queue is drained and each file is served from the buffer.
Recompile on change. Karma marks a changed file with a sha. When the plugin sees it, it queues the request and triggers a new compilation.

My first pull request

This was my first open-source project, and I was really happy the day another contributor sent me a pull request. It was #2, opened by @rich-j on February 7th, 2016, and merged the next day. It added sourcemap support through gulp-sourcemaps, because gulp-typescript ignores the sourcemap settings in tsconfig.json, and it made tsconfigPath relative to Karma’s basePath.

A few months later, @lelandmiller sent #3, merged on May 21st, 2016. It compared files by full path instead of file name, so two files with the same name in different folders stopped colliding.

Where it ended up

The last version on npm is 1.2.1, from June 19th, 2016. The repository got a few more commits until 2017 (a refactor and a fix for default compiler options), but they were never published. If you read master today, be aware it does not match what npm serves: index.js requires lodash.assignin while package.json lists lodash.assign, and the transformPath check throws when you pass an array.

The ecosystem moved on too: Karma’s own README now says “Karma is deprecated and is not accepting new features or general bug fixes”.

What I take from it

Two decisions still feel right to me: keeping compiled output in memory, so there is nothing to clean up, and letting tsconfig.json be the single source of truth for the compiler.

The code is about 200 lines and MIT licensed. If you are curious how a Karma plugin is wired together (the $inject array, the factory, the done callback), it is a quick read. Take a look on GitHub, and if you still have an old project running it, I would love to hear about it in the comments.

karma-typescript-preprocessor2: running TypeScript tests in Karma without temp files was originally published by Cleiton Loiola at Esta couve flor on November 11, 2015.

CALLs: how they use the stack (in Portuguese)

2009-04-02T00:00:00+00:00

English readers: this is my 2009 Portuguese translation of a beginner’s tutorial on how CALLs use the stack. The original is in English at woodmann.com, so go read it there.

/_//_                                                    
/`\//_|/_/ 
	_/ -.- .-.. .- -.-- __________________Assembly IA-32 
	---------------------------------------------
=-------- CALL's - Como elas fazem uso da Stack ----------= 
	--------------------------------------------- 

Introdução

Este pequeno artigo fala sobre procedures(funções) e como elas usam a pilha para passar variáveis. Ele poderá ser muito util principalmente para iniciantes. Eu não acho que crackers experientes terão muito proveito deste, porém se você não sabe como a pilha funciona, o que uma procedure faz e como os parametros são passados para ela, então este documento poderá ajuda-lo e muito. Eu também gostaria de avisar que todos nós somos responsáveis por nossas próprias ações. Eu não serei responsável por nenhum ato ilegal que seja tomado com o conhecimento providenciado aqui. Então, se ajeite em uma posição confortável divita-se :)

Ferramentas Necessárias

Tudo que você vai precisará será:

Algum espaço em seu celebro
Um drink; Um long island-ice tea dá para o gasto:
- 4 dedos de rum,
- 4 dedos de tequila,
- 4 dedos de whiskey,
- 4 dedos de vodka,
- Um pouco de suco de limão e preencha o resto com coca-cola. Uma delicia!

Porque usar procedures?

Eu tenho certeza que você já se perguntou o que é uma procedure. Qual sua utilidade? Imagine que você esta desen- volvendo um programa que tenha que escrever alguns caracteres na tela com uma certa frequencia por exemplo. Este texto não será o mesmo todas as vezes, mas a rotina, o código para tra- zer os caracteres a tela será sempre o mesmo.

Sendo assim, para fazer isso,você teria que escrever o mesmo código repetida vezes para cada mensagem que você desejasse exibir na tela, porém há uma maneira mais fácil. Apenas escreva o código uma vez, logo após lhe de um nome, uma referencia, como por exemplo “write_to_screen” e então, a parti de agora tudo o que você precisa fazer é chamar esta função. Mas espere, há outro problema: Nos temos diferente mensagens, então, nos iremos precisar de uma variável, algo como MSG que guardaria a mensagem que nos queremos imprimir e é claro nos teremos que passa-la a função. Esta tarefa se pareceria assim, caso fosse escrito em uma linguagem de alto nível como C++.

Exemplo:
	write_to_screen(LPSTR *msg);

Mas como seria em Assembly? Seria mais ou menos assim:

Exemplo:
	push 	MSG				;insiro MSG na pilha
	call	write_to_screen	;chamo a procedure

Este exemplo mostra dois tópicos que foram explicados neste tutorial. O que uma chamada a procedure é e porque você precisa passar variáveis para ela. Agora que você sabe para que serve uma procedure e o que ela tem de melhor, talvez você esteja confuso por causa do push, então nossa próxima etapa é falar um pouco mais sobre a pilha.

A PILHA

A pilha é uma estrutura que guarda valores. Estes valores sempre possuem 4 bytes (= 1 DWORD) de comprimento. Ela funciona em LIFO, sigla para Last In First Out que em português quer dizer O primeiro que entra é o ultimo a sair.

Isso quer dizer que um valor que foi recentemente colocado na pilha será removido primeiro. Há apenas dois comandos, instruções, para trabalhar com a pilha. estes são chamados push e pop. push insere um valor na pilha e pop remove este valor.

Exemplo:

push a		Pilha= {a}
push b		Pilha= {a,b}
push c		Pilha= {a,b,c}
pop eax		Pilha= {a,b} eax = c
pop ebx		Pilha= {a} eax = c; ebx = b
push eax   	Pilha= {a,c} eax = c; ebx = b
push 5		Pilha= {a,c,5} eax = c; ebx = b

Mas push e pop também fazem coisas adicionais. para entender, nós precisamos primeiro dá uma olhada para o registrador ESP (Apontador de pilha). Este registrador sempre terá uma referencia para o inicio da pilha. ou melhor o ultimo valor que foi colocado lá. a pilha sempre cresce de um valor alto de memória para um baixo, então se você incluir um valor ESP será subtraido por 4, se você remover um valor ESP será incrementado por 4.

Exemplo:

ESP = 7E0000; EAX = 01020304
: ...
:7DFFF0 12 34 56 78 00 00 00 00-00 00 00 00 00 00 00 00
:7E0000 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00

Este é o estado inicial da pilha preenchido com 78563421.

Agora se você inserir EAX na pilha você terá:

ESP = ESP - 4 = 7DFFF0                      ESP Aqui
: ...                                       |
:7DFFF0 00 00 00 00 00 00 00 00-00 00 00 00 04 03 02 01
:7E0000 12 34 56 78 00 00 00 00-00 00 00 00 00 00 00 00

Como você pode ver, o apontador de pilha é ajustado
e o valor é escrito na posição correta da memória.


pop ebx

Agora você removeu o valor:

ESP = ESP + 4 = 7E0000		EBX = 01020304

:7DFFF0 00 00 00 00 00 00 00 00-00 00 00 00 04 03 02 01
:7E0000 12 34 56 78 00 00 00 00-00 00 00 00 00 00 00 00
		|
		ESP aponta para cá.
		Note que o valor removido não foi modificado!

É claro qu evocê também pode adcionar ou subtrair algo de ESP. Vamos ver o que acontece se nós adcionarmos 1 a ESP

INC ESP ; ESP = ESP + 1 = 7E0001

:7DFFF0 00 00 00 00 00 00 00 00-00 00 00 00 04 03 02 01
:7E0000 12 34 56 78 00 00 00 00-00 00 00 00 00 00 00 00
		   |
		   ESP agora aponta para cá.

Vamos agora remover um valor:

pop EBX
ESP = ESP + 4 = 7E0005
:7DFFF0 00 00 00 00 00 00 00 00-00 00 00 00 04 03 02 01
:7E0000 12 34 56 78 00 00 00 00-00 00 00 00 00 00 00 00
					   |
					   ESP agora aponta para cá.

E EBX ficou com o valor 00785634. Note que após esta operação a pilha ficou “fora de ordem”, então você deve sempre apenas adcionar multiplos de 4 a ESP.

Agora que você sabe como a pilha funciona e o que ela faz, vamos analisar as procedures mais profundamente:

PROCEDURES - Suas entranhas

Como você viu, a pilha é um lugar ideal para guardar valores. Vamos agora dá uma nova olhada ao nosso pequeno primeiro exemplo que mostramos anteriormente. Agora com o nosso novo conheicmneot podemos entender o que acontece e té mesmo perceber um bug sério.

push MSG              ; Não irá funcionar, já que a pilha só pode guadar 4 bytes!
call write_to_screen  ; então a procedure irá ler o valor de MSG da pilha

Fica claro que termos que passar o endereço de MSG ao invés de seu valor. Nos podemos fazer isso facilmente com com o comando(instrução) LEA EAX, MSG. AX agora possui o endereço de MSG. Logo após nos o inserimos com PUSH EAX. Vamos transcrever este processo e ver o que realmente esta acontecendo:

Antes do PUSH:

 dados: EAX = 00000000
	  :00450000 31 32 33 34 35 36 37 38-39 30 00 00 00 00 00 00   1234567890......

 pilha: ESP = 007E0000
	  :007DFFF0 00 00 00 00 00 00 00 00-00 00 00 00 04 03 02 01
	  :007E0000 12 34 56 78 00 00 00 00-00 00 00 00 00 00 00 00

Depois do PUSH:

	 :00412l00 LEA EAX, MSG  ; nada muda exeto que EAX agora vale 00450000
	 :00412104 push eax	  ; os dados continuam no lugar, porém a pilha muda
          ; pilha: ESP = 007DFFFC                         ............ address
          ;      :007DFFF0 00 00 00 00 00 00 00 00-00 00 00 00 00 00 45 00
          ;      :007E0000 12 34 56 78 00 00 00 00-00 00 00 00 00 00 00 00
			     :00412105 call write_to_screen
          ; insere a sequqencia na tela.
		  ; short break ---
			     :00412110 próxima instrução...

Mas o que a instrução call irá fazer? Ela irá fazer algo muito importante. Ela irá automáticamente salvar o endereço dá próxima instrução na pilha. Mas porque? Porque esta é uma chamada a procedure e não um JMP e depois que a procedure tiver feito seu trabalho, o programa deve continuar de onde parou antes da rotinar se chamada. Ou seja, isto é tudo que call é. Ser responsável em salvar o endereço da próxima instrução na pilha para que o PC saiba onde por onde prosseguir. Vamos ver como a pilha se parece depois de encontrar uma instrução call

Exemplo:

	pilha: ESP = 007DFFF8             ........... endereço de retorno salvo na stack
		 :007DFFF0 00 00 00 00 00 00 00 00-10 21 41 00 00 00 45 00
		 :007E0000 12 34 56 78 00 00 00 00-00 00 00 00 00 00 00 00

e o controle de execução pulará para onde call aponta. Sendo assim, sempre que você entra em uma procedure você tem no topo da pilha o endereço de retorno. Agora existe apenas um questão a resolver: Como a procedure ler os dados da pilha?

Lendo dados da stack

Esta é a parte mais interessante e há várias formas de se fazer, a primeira possibilidade é retirar o valor de retorno usando pop e salvar em algum lugar, depois retirar o valor dos dados restantes também usando pop e depois reincluir usando push o endereço de retorno. Esta tarefa pode ser complicada especiamente se você tem muitos valores em uma chamada enorme, o que forçará você a salvar os valores novamente na stack por que você não pode guarda-los todos em um registrador de dados. Pode funcionar para pequenas chamadas com poucos parametros.

Então qual seria outro jeito? É fácil, basta você salvar EBP na stack e depois guadar o endereço da stack dentro de EMP usando a instrução mov EBP, ESP agora você pode endereçar todas as variáveis que são relativas a EBP através de mov eax, [EBP+20], desta forma você terá maior controle da stack, você também poderá continuar usando a stack normalmente na chamada tendo que apenas ter certeza que o valor de retorno estará no topo da stack que você salvou em EBP quando você sair da chamada.

Vamos olhar este exemplos simples que eu extrai de um MP3 CD Maker com o Win32DASM:

beginning of the call
:004301A1 push ebp                    ; ebp foi salvo
:004301A2 mov ebp, esp                ; agora ebp aponta para o top da stack
									; você não precisa mais se preocupar com pushes e pops
									; por que você acessa os paramentros usando EBP como referencia agora
:004301A4 mov eax, dword ptr [ebp+08] ; é igual a pop eax, pop eax mas você perderia o valor de retorno
:004301A7 push esi                    ; salva outro valor
.
. call routine unimportant.
. leaving the call
:004301FF pop esi                     ; recupera  esi
:00430200 pop ebp                     ; recupera o ponteiro de base original
:00430201 ret                         ; agora o endereço de retorno esta no topo da stack

Dica Você pode ver algo como mov eax, [ebp+10], agora você renomear a variável [ebp+10] por que ela sempre apontará para o mesma chamada. Em uma chamada protegida com nome e serial, por exemplo, você poderia dizer que [ebp+10] e o nome e [ebp+14] é o serial. Isso fará com que programas e especialmente deadlistings muito mais fáceis de serem lidas. Agora com todo este conhecimento você já pode dá uma olhada em alguns codesnippets extraídos com o Win32DASM.

Tradução por Cleiton Gomes em 02/04/2009, artigo original: http://www.woodmann.com/krobar/beginner/168.htm

.EOF

CALLs: how they use the stack (in Portuguese) was originally published by Cleiton Loiola at Esta couve flor on April 02, 2009.

Esta couve flor

magic-cursor-wand: Give Your Pointer a Wand

What it does

Start in one minute

With a script tag

With a bundler

Make it yours

A few technical highlights

Try it

hurl-orchestra: AI first API automation testing

Why I built it

How it looks

The graph, in plain words

A failure stops only its branch

Why “AI first”

Under the hood

Where it went next

Try it

Understanding the Heart of the STM32: The Clock System

The STM32 Clock Sources

Let’s get to know each one of them:

The Clock Path: From Start to End

Special Clocks:

Configuring Everything in Code (SystemInit)

mini-painel: A Hobby Log About Talking to a Tiny USB Screen in C

It started with six bytes

What works today

A few details from under the hood

Building it

Using the library

What I’m still exploring

Try it

Free tools for embedded development

Arm Embedded Tool Chain

GNU/Linux

Windows

Mac

OpenOCD

GNU/Linux

Windows

Mac

GNU Make

Linux and Mac

Windows

CMSIS

Conclusion

Using GNU Make as your Dotfiles manager

The motivation

The GNU Make for rescue

The Makefile

Conclusion

MiniReact: a tiny React-like library to see what happens under the hood

What you get

Two words before we start

Writing a component

Two differences from React

How it works: three phases

Constructing

Patching

Unmounting

Events without leaking listeners

Tests

Running it

Conclusion

The small gotcha that made me feel I know something

GeoLocation: a take-home challenge built with React, Redux and Sagas

The challenge

The stack

How the data flows

Validating the input

A design decision I left in the code

What I left out

Update, September 2026: running it today

Conclusion

Introduction to async and await (in Portuguese)

Introduzindo as palavras-chaves

Awaitables

Tipos de retorno

Retornando valores

Contexto

Configuring Everything in Code (`SystemInit`)