Category Archives: Sonar

Auto Added by WPeMatico

Introducing sonarwhal v1: The linting tool for the web

Just over one year ago, we started working on a best practices tool for the web called sonarwhal—a customizable, open-source linting tool, built for modern web developer workflows. Today, we are announcing the release of its first major version. With today’s launch, we’d like to talk you through a look back at how sonarwhal started, and the journey to v1 over the past few months.
It all started with feedback from web developers, partners, and from our own experiences building for the web. The web platform is becoming richer at a faster pace than ever before: we now have web experiences we couldn’t even imagine a few years back. Sites can work offline, push notifications (even when the user is not visiting the site), and even run code at native speeds with WebAssembly. On top of keeping up to date, developers also need to know about many other things like accessibility, security, performance, bundling, transpiling, etc.
Every day, we see sites that have a great architecture, built with the latest libraries and tools, but that don’t use the right cache policy for their static assets, or that don’t compress everything they should, or with common security flaws—and that’s just scratching the surface. The web is complex and it can be easy to miss something at any point during the development process.
Being a web developer can be overwhelming
What could we do to help web developers make their sites faster, more secure and accessible while at the same time making sure they remain interoperable cross browser? And that’s how all started. Picking the name was the easiest part: the team loves narwhals and they have one of the best echolocation beams in nature (or sonars) so it was an obvious choice.
We knew from the beginning that we wanted sonarwhal to be built by and for the web community. We didn’t want to indoctrinate our personal opinions into sonarwhal’s rules. We wanted sonarwhal to be backed by deep research, remain neutral, and allow contributions from any individual or company. Thus, we decided to make sonarwhal an open source project as part of the JS Foundation.
Since then, we’ve kept ourselves busy listening to your feedback and implementing many of the goals we had back then, while adding some new ones.
When creating a new rule, we follow the 80/20 principle:
80% of the time is research and 20% is coding
If there’s one thing we are most proud of, it’s the extensive research we do on each subject and how deep the rules go to check that everything is as it should be.
Just to give you some examples:
The http-compression rule will perform several requests for each resource with different headers and check the content to validate the server is actually respecting them. E.g.: When resources are requested uncompressed, does the server actually respect what was requested and serve them uncompressed? Is the server doing User-Agent sniffing instead of relying on the Accept-Encoding header of the request? Is the server compressing resources using Zopfli when requests are made advertising support for gzip compression?
The web manifest rules are also interesting. Does the web manifest point to an image? Does that image exist? Does the image meet the recommended resolution and file size? Does it have the right format to be used by any browser? Is the name of the web application short enough to be displayed on all platforms?
The web is full of lies (starting with the user-agent string). Just because a file ends with .png and has content-type: image/png doesn’t mean it’s a PNG. It could very well be a JPEG file, or something completely different. And the same goes for every downloaded resource. The content-type rule will look at the bytes of the resources and verify. the server is actually serving what it says it is, and where applicable, that is specifying the proper charset.
And the list goes on…
More than 30 rules in 6 categories (and counting!)
sonarwhal validates many different things: from accessibility and content types, to verifying your JavaScript libraries don’t have any known vulnerabilities and that you are using SRI to validate that no one has tampered with the code.
Some of the issues require developers to change their code, but others require tweaking to the server configurations. Changing the configurations might not be obvious, especially when targeting only certain resource types, or newer tools and techniques such as Brotli compression, which may not be as thoroughly documented. To make the developer experience easier, we’ve also added examples for Apache and IIS for the rules that require it.
Get started testing with our online scanner
sonarwhal runs on top of Node.js and is distributed via npm. But what happens if you want to check a site using your mobile phone? Or maybe your IT administrator doesn’t allow you to install any tool.
We needed a way to scale and make sonarwhal available anywhere with an internet connection.In November of last year we launched the online version. Since then, more than 160,000 URLs have been analyzed.
Each result page has its own permalink to allow you to go back to it later, or share it with anyone.The code for the online scanner is available on GitHub.
Scan results for https://example.com
Configure sonarwhal to your needs
We think tools should be helpful and should stay out of your way. We can tell you what we think is important, but at the end of the day, you are the one that best understands what you are building and what requirements you have.
We build sonarwhal with strong defaults, but with the flexibility to let you decide what rules are relevant to your project, what URLs should be ignored, what browser you want to use—essentially, we want everything to be configurable.
To make it easier to reuse configurations, you can now extend from one or more and tweak the properties you want.
For example, to use the configuration “web-recommended” you just have to:
npm install @sonarwhal/configuration-web-recommended
And tell your .sonarwhalrc file to extend from it:
{
  "extends": ["web-recommended"]
}
If you want to tweak it, you can do this:
{
  "extends": ["web-recommended"],
  "ignoredUrls": [{
    "domain": ".*.domain1.com/.*",
    "rules": ["*"]
  }],
  "rules": {
    "highest-available-document-mode: ["error", {
      "requireMetaTag": true
    }]
  }
}
The above snippet will use the defaults of “web-recommended”, ignore all resources that match the regular expression .*.domain1.com/.* and enforce the X-UA-Compatible meta tag instead of the header.
Without doubt, one of our favorite features is the adaptability of the rules. Depending on the browsers you want to support, some rules will adapt their feedback telling you the best approach for your specific case. We believe this is really important because not everybody gets to develop for the latest browser versions.
Easily extend sonarwhal with parsers to analyze files such as config files
Catching issues early in the development cycle is usually better than when the project is already shipped. To help with this we created the “local connector” that allows you to validate the files you are working with.
Building a website these days usually requires more than just writing HTML, JavaScript and CSS files. Developers use task managers, bundlers, transpilers, and compilers to generate their code. And each one of these needs to be configured, which in some cases is not easy. To tackle this problem, we came up with the concept of a parser. A parser understands a resource format and is capable of emitting information about it so rules can use it.
Parsers are a powerful concept. They allow sonarwhal to be expanded to support new scenarios we couldn’t imagine when we first started the project. In our ca se, we’ve started creating parsers for config files of the most popular tools used during the build process ( tsconfig.json, .babelrc, and webpack.config.js so far) and rules related to them:
By default TypeScript output is ES3 compatible, but maybe you don’t need to go all the way down and could be using ES5, ES6, etc. Using the information of your browserslist, we’ll tell you what target you should have in your tsconfig.json file.
 If you are using webpack, you should have “modules”: false in your .babelrc file to make sure you have better tree shaking and thus, less generated code.
These are just some relatively basic examples of what’s possible. Parsers allow you to create rules for virtually anything. For example, someone could create a parser that understands the metadata of image files and then a rule that checks that all the images have a valid “copyright status.”

sonarwhal analyzing configuration files for webpack and TypeScript
sonarwhal v1 is now available! Go get it while it’s fresh!
As you can see, we’ve been busy! After all that, we’re finally ready to announce the first major version of sonarwhal!
While this is a big milestone for us, it doesn’t mean we are going to remain idle. Indeed, now that GitHub organization projects can be public we’ve opened up ours so you can know the project’s priorities and what we are working on.
Some of the things we are more excited about are:
New rules: you can expect more rules around security, performance, PWA, and development very soon.
User actions for the browser: sometimes the page or scenario you need to test requires a bit of interaction to get to it. We are looking in ways to allow you to control the browser before a scan.
Custom configuration for the online scanner: the user should be capable of deciding everything, even when using the online scanner.
Notifications for the online scanner: some websites take longer than others to scan and is easy to forget you have something running on a background tab. We will add opt-in notifications to let you know when all the results are gathered.
sonarwhal is completely open source and community driven. Your feedback is really appreciated and helps us prioritize this list. And if you want to help developers all around the world, join us!
– Antón Molleda, Senior Program Manager, Microsoft Edge

sonar: Linting the web forward

In addition to building a great browser and dependable web platform for Windows, we’re passionate about empowering developers to build great websites. With that goal in mind, we launched modern.IE in 2013 featuring a static scan tool to detect optimizations for old versions of IE, outdated libraries, missing prefixes, and more. The web has moved on since then, and it’s time to update our tools accordingly!
Introducing sonar
Today, we are excited to announce the next evolution of the static scan tool: sonar, a new linting tool and site scanner for the modern web.

sonar brings many improvements compared to previous scanners: execution of websites code instead of static analysis, a more flexible and modernized set of rules, parallel test execution, integration with other services, a completely open source code base from day one, and more. Additionally, sonar can also be used as a command line tool (CLI) that you can integrate directly into your local web development workflows.
Web development is more than HTML, JavaScript, and CSS: developers are expected to have a grasp of accessibility, performance, security, emerging standards, and more, all while refreshing this knowledge every few months as the web evolves.
Web development can be complex.
Linting the web forward
Simply put, the web is complex, and we want sonar to make it a bit easier for you to write great websites. To make sure that sonar can be helpful not only now, but in the future, we started with a set of guiding principles before we wrote a single line of code.
Put the user at the center
Rather than just telling developers what was wrong, sonar had to also say why. It is important to know the reason for an issue so developers can decide if that really applies to their work. The requirements from website to website can change a lot, for example, an intranet website and an online shopping experience will have vastly different needs. Therefore, sonar should also be easy to use, configure, and expand.
Build for the community’s best interests
The web belongs to everyone, and this project should too. Everything had to be open sourced since the beginning, but that wasn’t enough―we wanted to go even further and make it easier for the web community to get involved, and remove any possible doubt that this project has the community’s best interest in mind. For that reason, we decided to donate the project to the JS Foundation early during the summer.
Collaborate with existing tools and services
sonar should avoid reinventing the wheel, instead leveraging and integrating existing tools and services that help developers build for the web. We are happy to say that sonar now integrates with aXe Core, AMP validator, snyk.io, SSL Labs, and Cloudinary.

You can hear more about sonar’s history and guiding principles in our session at Microsoft Edge Web Summit.
sonar today, and what’s next
We’ve come a long way since we wrote down those principles a few months ago: sonar is now available as an open sourced command line utility, built on node, that you can install via npm. Additionally, it has an open-source online service, deployed on top of Azure, using docker containers, that can scan any publicly available website. sonar’s rules are backed by a collection of best practices for the web, with links to more detailed documentation that keeps growing with each new rule.
But this is just the beginning. We’re hard at work on a backlog of exciting features for future releases, such as:
A plug-in for Visual Studio Code: We want sonar to help you write better websites, and what better moment than when you are in your editor.
Configuration options for the online service: As we fine tune the infrastructure, the rule configuration for our scanner is locked, but we look forward to adding customization options here in the near future.
New rules for a variety of areas like performance, accessibility, security, Progressive Web Apps, and more.
If you are excited about sonar, making a better web, and want to contribute, we have a few issues where you might be able to help. Also, don’t forget to check the rest of the sonarwhal GitHub organization. PRs are always welcome and appreciated!
Take a few moments to try the sonar scanner and the CLI, and let us know what you think at @narwhallnellie on Twitter or in the comments below!
– Antón Molleda, Senior Program Manager, Microsoft Edge
 

Microsoft Edge Web Summit 2017 recordings are now available on Channel 9

Last week we welcomed hundreds of local developers and thousand of online viewers to our third annual Microsoft Edge Web Summit! Videos and slides from each session are now available to stream or download on Channel 9.
Learn about what’s new in EdgeHTML 16 in the keynote at Microsoft Edge Web Summit 2017.
Our sessions will bring you up to date on what’s in store for EdgeHTML 16, including learning how to use new and updated features like CSS Grid Layout, object-fit and object-position, WebVR, and the Web Payments API.

Learn about how to build faster websites with a fast and furious tour of web performance in the real world, and how to keep your development and testing on track with sonar, a new open-source, community-owned linting tool for the web. And make sense of the always-evolving web app landscape while blending the best of web and native with Progressive Web Apps.

Or go on a deep dive into the inner workings of the browser, to learn how we’re constantly rebuilding Microsoft Edge to be more secure, more accessible, and faster than ever, with every release we ship.

That’s just the beginning – there’s lots more to see on Channel 9, and we’ll have more to share about these topics and more in the coming weeks right here on the Microsoft Edge Dev Blog.
Thanks for joining us at Microsoft Edge Web Summit 2017 – we can’t wait to see you next year!
— Kyle Pflug, Senior Program Manager, Microsoft Edge