Flycheck versus Flymake¶

This article provides information about Flycheck compares to the built-in Flymake mode. It does not consider third-party extensions such as flymake-easy or flymake-cursor, but references them at appropriate places.

We aim for this comparison to be neutral and complete, but do not provide any guarantee for completeness or correctness of the following information. Moreover, we consider Flycheck superior to Flymake in all aspects. As such, you may find this page biased towards Flycheck. Please excuse this as well as any factual mistake or lack of information. Please suggest improvements.

Important

This comparison was written around the time Emacs 24.5 was released, and only updated infrequently since then. Flycheck has changed and hopefully improved meanwhile, and Flymake may have done so as well. As such parts of this article may be outdated and have become incorrect by now. Likewise screenshots that show particular behaviour of Flycheck or Flymake have aged; the corresponding features of Flycheck and Flymake may look different now, or have gone altogether.

Please report any incorrectness and any inconsistency you find, and feel free to edit this page and improve it.

Overview¶

This table intends to give an overview about the differences and similarities between Flycheck and the default install of Flymake. It is not a direct comparison to third-party extensions like flymake-easy or flymake-cursor. For a more comprehensive look compared to those extensions, please read the details in the main article and the footnotes.

Please do not use this table alone to make your personal judgment. Read the detailed review in the following sections, too, at least with regards to the features you are interested in.

	Flycheck	Flymake
Supports Emacs versions	24.3	22+
Built-in	no [1]	yes
Enables automatically if possible	yes	no
Checks after	save, newline, change	newline, change
Checks in background	yes	yes
Automatic syntax checker selection	By major mode and custom predicates	By file name patterns [2]
Manual syntax checker selection	yes	no
Multiple syntax checkers per buffer	yes	no [3]
Supported languages	>40	~5 [4]
Checking remote files via Tramp	said to work, but not officially supported [5]	partly?
Definition of new syntax checkers	Single declarative function/macro	Function definition and various variables [6]
Functions as syntax checkers	yes	no [7]
Error levels	errors, warnings, informational, custom levels	errors, warnings
Error identifiers	yes	no
Error parsing	Regular expressions, custom parsers for structured formats (XML, JSON, etc.)	Regular expressions
Multiline error messages	yes	no
Error highlighting in buffers	yes	yes
Fringe icons for errors	yes	yes (Emacs 24.1+)
Error message display	Tooltip, echo area, fully customizable	Tooltip only [8]
List of all errors	yes	no
Resource consumption	low	high
Unit tests	all syntax checkers, large parts of internals	none?

Detailed review¶

Relation to Emacs¶

Flymake is part of GNU Emacs since GNU Emacs 22. As such, contributions to Flymake are subject to the FSF policies on GNU projects. Most notably, contributors are required to assign their copyright to the FSF by signing a contributor agreement.

Flycheck is not part of GNU Emacs, and is unlikely to ever be (see issue 801). However, it is free software as well, and publicly developed on the well-known code hosting platform Github. Contributing to Flycheck does not require a copyright assignments.

Enabling syntax checking¶

Flymake is not enabled automatically for supported languages. It must be be enabled for each mode individually and carefully, because it does not deal well with unavailable syntax checker tools. In a GUI frame, it signals errors in GUI dialogs. In a TTY frame, it does not signal any error at all, but instead silently hangs. The same occurs when a syntax checker tool becomes unavailable after Flymake Mode is enabled (for instance, because the underlying tool was uninstalled).

Flymake showing a GUI dialog to inform that a syntax checker tool is not available

The third-party library flymake-easy provides an alternate way to enable Flymake Mode, which gracefully handles unavailable syntax checkers. It does not check whether the tool still exists before a syntax check, though, and thus does still exposes above behavior when a tool becomes unavailable after the mode was enabled.

Flycheck provides a global mode global-flycheck-mode, which enables syntax checking in every supported language. If a syntax checking tool is not available Flycheck fails gracefully, does not enable syntax checking, and just indicates the failure in the mode line.

Syntax checkers¶

Flymake supports Java, Makefiles, Perl, PHP, TeX/LaTeX and XML. Notably, it does not support Emacs Lisp. However, there are many recipes for other languages on the Flymake page in the EmacsWiki and many extension packages for other languages in the popular ELPA archive MELPA.

Flycheck provides support for over 40 languages with over 70 syntax checkers, most of them contributed by the community. Notably, Flycheck does not support Java and Makefiles.

Definition of new syntax checkers¶

Flymake does not provide a single function to define a new syntax checker. Instead, one has to define an “init” function, which returns the command, and add this function to flymake-allowed-file-name-masks. Additionally, one has to add the error patterns to flymake-err-line-patterns. As such, defining a syntax checker is difficult for users who are not familiar with Emacs Lisp. flymake-easy provides an easier way to define new syntax checkers, though.

Flycheck provides a single function flycheck-define-checker to define a new syntax checker. This function uses a declarative syntax which is easy to understand even for users unfamiliar with Emacs Lisp. In fact most syntax checkers in Flycheck were contributed by the community.

For example, the Perl checker in Flymake is defined as follows:

(defun flymake-perl-init ()
  (let* ((temp-file   (flymake-init-create-temp-buffer-copy
                       'flymake-create-temp-inplace))
         (local-file  (file-relative-name
                       temp-file
                       (file-name-directory buffer-file-name))))
    (list "perl" (list "-wc " local-file))))

(defcustom flymake-allowed-file-name-masks
  '(;; …
    ("\\.p[ml]\\'" flymake-perl-init)
    ;; …
    ))

(defvar flymake-err-line-patterns
  (append
   '(;; …
     ;; perl
     ("\\(.*\\) at \\([^ \n]+\\) line \\([0-9]+\\)[,.\n]" 2 3 nil 1)
     ;; …
     )
   ;; …
   ))

Whereas Flycheck’s definition of the same checker looks like this:

(flycheck-define-checker perl
  "A Perl syntax checker using the Perl interpreter.

See URL `http://www.perl.org'."
  :command ("perl" "-w" "-c" source)
  :error-patterns
  ((error line-start (minimal-match (message))
          " at " (file-name) " line " line
          (or "." (and ", " (zero-or-more not-newline))) line-end))
  :modes (perl-mode cperl-mode))

Functions as syntax checkers¶

Flymake cannot check a buffer with a custom Emacs Lisp function.

Flycheck provides the flycheck-define-generic-checker function to define a syntax checker based on an arbitrary Emacs Lisp function. Flycheck supports synchronous as well as asynchronous functions, and provides simple callback-based protocol to communicate the status of syntax checks. This allows Flycheck to use persistent background processes for syntax checking. For instance, flycheck-ocaml uses a running Merlin process to check OCaml buffers. This is much easier and faster than invoking the OCaml compiler.

Customization of syntax checkers¶

Flymake does not provide built-in means to customize syntax checkers. Instead, when defining a new syntax checker the user needs to declare customization variables explicitly and explicitly check their value in the init function.

Flycheck provides built-in functions to add customization variables to syntax checkers and splice the value of these variables into the argument list of a syntax checking tool. Many syntax checkers in Flycheck provide customization variables. For instance, you can customize the enabled warnings for C with flycheck-clang-warnings. Flycheck also tries to automatically find configuration files for syntax checkers.

Executables of syntax checkers¶

Flymake does not provide built-in means to change the executable of a syntax checker.

Flycheck implicitly defines a variable to set the path of a syntax checker tool for each defined syntax checker and provides the interactive command flycheck-set-checker-executable to change the executable used in a buffer.

Syntax checker selection¶

Flymake selects syntax checkers based on file name patterns in flymake-allowed-file-name-masks. Effectively this duplicates the existing logic Emacs uses to choose the right major mode, but lacks its flexibility and power. For instance, Flymake cannot pick a syntax checker based on the shebang of a file.

Flycheck uses the major mode to select a syntax checker. This reuses the existing sophisticated logic Emcas uses to choose and configure major modes. Flycheck can easily select a Python syntax checker for a Python script without file extension, but with proper shebang, simply because Emacs correctly chooses Python Mode for such a file.

Custom predicates¶

Flymake does not allow for custom predicates to implement more complex logic for syntax checker selection. For instance, Flymake cannot use different syntax checkers for buffer depending on the value of a local variable.

However, flymake-easy patches Flymake to allow for custom syntax checkers per buffer. This does not happen automatically though. The user still needs to explicitly register a syntax checker in a major mode hook.

Flycheck supports custom predicate function. For instance, Emacs uses a single major mode for various shell script types (e.g. Bash, Zsh, POSIX Shell, etc.), so Flycheck additionally uses a custom predicate to look at the value of the variable sh-shell in Sh Mode buffers to determine which shell to use for syntax checking.

Manual selection¶

Flymake does not provide means to manually select a specific syntax checker, either interactively, or via local variables.

Flycheck provides the local variable flycheck-checker to explicitly use a specific syntax checker for a buffer and the command flycheck-select-checker to set this variable interactively.

Multiple syntax checkers per buffer¶

Flymake can only use a single syntax checker per buffer. Effectively, the user can only use a single tool to check a buffer, for instance either PHP Mess Detector or PHP CheckStyle. Third party extensions to Flycheck work around this limitation by supplying custom shell scripts to call multiple syntax checking tools at once.

Flycheck can easily apply multiple syntax checkers per buffer. For instance, Flycheck will check PHP files with PHP CLI first to find syntax errors, then with PHP MessDetector to additionally find idiomatic and semantic errors, and eventually with PHP CheckStyle to find stylistic errors. The user will see all errors reported by all of these utilities in the buffer.

Errors¶

Error levels¶

Flymake supports error and warning messages. The pattern of warning messages is hard-coded in Emacs 24.3, and only became customizable in upcoming Emacs 24.4. The patterns to parse messages are kept separate from the actual syntax checker.

Flycheck supports error, warning and info messages. The patterns to parse messages of different levels are part of the syntax checker definition, and thus specific to each syntax checker. Flycheck allows to define new error levels for use in custom syntax checkers with flycheck-define-error-level.

Error identifiers¶

Flymake does not support unique identifiers for different kinds of errors.

Flycheck supports unique identifiers for different kinds of errors, if a syntax checker provides these. The identifiers appear in the error list and in error display, and can be copied independently, for instance for use in an inline suppression comment or to search the web for a particular kind of error.

Error parsing¶

Flymake parses the output of syntax checker tools with regular expressions only. As it splits the output by lines regardless of the regular expressions, it does not support error messages spanning multiple lines (such as returned by the Emacs Lisp byte compiler or by the Glasgow Haskell Compiler).

flymake-easy overrides internal Flymake functions to support multiline error messages.

Flycheck can use regular expressions as well as custom parsing functions. By means of such functions, it can parse JSON, XML or other structured output formats. Flycheck includes some ready-to-use parsing functions for well-known output formats, such as Checkstyle XML. By parsing structured output format, Flycheck can handle arbitrarily complex error messages. With regular expressions it uses the error patterns to split the output into tokens and thus handles multiline messages just as well.

Error message display¶

Flymake error message in tooltip

../_images/flycheck-tooltip-and-echo-area.png

Flycheck error message in tooltip and echo area

In GUI frames, Flymake shows error messages in a tool tip, if the user hovers the mouse over an error location. It does not provide means to show error messages in a TTY frame, or with the keyboard only.

The third-party library flymake-cursor shows Flymake error messages at point in the echo area, by overriding internal Flymake functions.

Flycheck shows error message tool tips as well, but also displays error messages in the echo area, if the point is at an error location. This feature is fully customizable via flycheck-display-errors-function.

Error list¶

Flymake does not provide means to list all errors in the current buffer.

Flycheck can list all errors in the current buffer in a separate window. This error list is automatically updated after each syntax check, and follows the focus.

Listing all errors in the current buffer

Resource consumption¶

Syntax checking¶

Flymake starts a syntax check after every change, regardless of whether the buffer is visible in a window or not. It does not limit the number of concurrent syntax checks. As such, Flymake starts many concurrent syntax checks when many buffers are changed at the same time (e.g. after a VCS revert), which is known to freeze Emacs temporarily.

Flycheck does not conduct syntax checks in buffers which are not visible in any window. Instead it defers syntax checks in such buffers until after the buffer is visible again. Hence, Flycheck does only start as many concurrent syntax checks as there are visible windows in the current Emacs session.

Checking for changes¶

Flymake uses a separate timer (in flymake-timer) to periodically check for changes in each buffer. These timers run even if the corresponding buffers do not change. This is known to cause considerable CPU load with many open buffers.

Flycheck does not use timers at all to check for changes. Instead it registers a handler for Emacs’ built-in after-change-functions hook which is run after changes to the buffer. This handler is only invoked when the buffer actually changed and starts a one-shot timer to delay the syntax check until the editing stopped for a short time, to save resources and avoid checking half-finished editing.

Unit tests¶

Flymake does not appear to have a test suite at all.

Flycheck has unit tests for all built-in syntax checkers, and for large parts of the underlying machinery and API. Contributed syntax checkers are required to have test cases. A subset of the test suite is continuously run on Travis CI.

Footnotes

[1]	Flycheck is unlikely to ever become part of Emacs, see issue 801.

[2]	The 3rd party library flymake-easy allows to use syntax checkers per major mode.

[3]	Various 3rd party packages thus use custom shell scripts to call multiple syntax checking tools at once.

[4]	However, the Flymake page in the EmacsWiki provides recipes for many other languages, although of varying quality. Furthermore, the popular ELPA archive MELPA provides many packages which add more languages to Flymake.

[5]	See for instance this comment.

[6]	flymake-easy provides a function to define a new syntax checker, which sets all required variables at once.

[7]	flymake-easy overrides internal functions of Flymake to add support for multiline error messages.

[8]	The 3rd party library flymake-cursor shows Flymake error messages at point in the echo area.