在线时间:8:00-16:00
迪恩网络APP
随时随地掌握行业动态
扫描二维码
关注迪恩网络微信公众号
开源软件名称:gjtorikian/html-proofer开源软件地址:https://github.com/gjtorikian/html-proofer开源编程语言:Ruby 61.0%开源软件介绍:HTMLProoferIf you generate HTML files, then this tool might be for you! Project scopeHTMLProofer is a set of tests to validate your HTML output. These tests check if your image references are legitimate, if they have alt tags, if your internal links are working, and so on. It's intended to be an all-in-one checker for your output. In scope for this project is any well-known and widely-used test for HTML document quality. A major use for this project is continuous integration -- so we must have reliable results. We usually balance correctness over performance. And, if necessary, we should be able to trace this program's detection of HTML errors back to documented best practices or standards, such as W3 specifications. Third-party modules. We want this product to be useful for continuous integration so we prefer to avoid subjective tests which are prone to false positive results, such as spell checkers, indentation checkers, etc. If you want to work on these items, please see the section on custom tests and consider adding an implementation as a third-party module. Advanced configuration. Most front-end developers can test their HTML using our command line program. Advanced configuration will require using Ruby. InstallationAdd this line to your application's Gemfile:
And then execute:
Or install it yourself as:
NOTE: When installation speed matters, set What's tested?Below is a mostly comprehensive list of checks that HTMLProofer can perform. Images
Links
Scripts
Favicon
OpenGraph
UsageYou can configure HTMLProofer to run on:
It can also run through the command-line. Using in a script
Here's an example: require 'html-proofer'
require 'html/pipeline'
require 'find'
# make an out dir
Dir.mkdir("out") unless File.exist?("out")
pipeline = HTML::Pipeline.new [
HTML::Pipeline::MarkdownFilter,
HTML::Pipeline::TableOfContentsFilter
], gfm: true
# iterate over files, and generate HTML from Markdown
Find.find("./docs") do |path|
if File.extname(path) == ".md"
contents = File.read(path)
result = pipeline.call(contents)
File.open("out/#{path.split("/").pop.sub('.md', '.html')}", 'w') { |file| file.write(result[:output].to_s) }
end
end
# test your out dir!
HTMLProofer.check_directory("./out").run Checking a single fileIf you simply want to check a single file, use the HTMLProofer.check_file('/path/to/a/file.html').run Checking directoriesIf you want to check a directory, use HTMLProofer.check_directory('./out').run If you want to check multiple directories, use HTMLProofer.check_directories(['./one', './two']).run Checking an array of linksWith HTMLProofer.check_links(['https://github.com', 'https://jekyllrb.com']).run Swapping informationSometimes, the information in your HTML is not the same as how your server serves content. In these cases, you can use run_proofer(file, :file, swap_urls: { %r{^https//example.com}: 'https://website.com' }) In this case, any link that matches the A similar swapping process can be done for attributes: run_proofer(file, :file, swap_urls: { 'img': [['src', 'srcset']] }) In this case, we are telling HTMLProofer that, for any Using on the command-lineYou'll also get a new program called Pass in options through the command-line as flags, like this: htmlproofer --extensions .html.erb ./out Use Special cases for the command-lineFor options which require an array of input, surround the value with quotes, and don't use any spaces. For example, to exclude an array of HTTP status code, you might do: htmlproofer --http-status-ignore "999,401,404" ./out For something like htmlproofer --url-ignore "/www.github.com/,/foo.com/" ./out Since htmlproofer --swap-urls "wow:cow,mow:doh" --extensions .html.erb --url-ignore www.github.com ./out Some configuration options--such as
Adjusting for a |
Option | Description | Default |
---|---|---|
allow_hash_href |
If true , assumes href="#" anchors are valid |
true |
allow_missing_href |
If true , does not flag a tags missing href . In HTML5, this is technically allowed, but could also be human error. |
false |
assume_extension |
Automatically add specified extension to files for internal links, to allow extensionless URLs (as supported by most servers) | .html |
checks |
An array of Strings indicating which checks you want to run | ['Links', 'Images', 'Scripts'] |
check_external_hash |
Checks whether external hashes exist (even if the webpage exists) | true |
check_sri |
Check that <link> and <script> external resources use SRI |
false |
directory_index_file |
Sets the file to look for when a link refers to a directory. | index.html |
disable_external |
If true , does not run the external link checker |
false |
enforce_https |
Fails a link if it's not marked as https . |
true |
extensions |
An array of Strings indicating the file extensions you would like to check (including the dot) | ['.html'] |
ignore_files |
An array of Strings or RegExps containing file paths that are safe to ignore. | [] |
ignore_empty_mailto |
If true , allows mailto: href s which do not contain an email address. |
false |
ignore_missing_alt |
If true , ignores images with empty/missing alt tags |
false |
ignore_status_codes |
An array of numbers representing status codes to ignore. | [] |
ignore_urls |
An array of Strings or RegExps containing URLs that are safe to ignore. This affects all HTML attributes, such as alt tags on images. |
[] |
log_level |
Sets the logging level, as determined by Yell. One of :debug , :info , :warn , :error , or :fatal . |
:info |
only_4xx |
Only reports errors for links that fall within the 4xx status code range. | false |
root_dir |
The absolute path to the directory serving your html-files. | "" |
swap_attributes |
JSON-formatted config that maps element names to the preferred attribute to check | {} |
swap_urls |
A hash containing key-value pairs of RegExp => String . It transforms URLs that match RegExp into String via gsub . |
{} |
In addition, there are a few "namespaced" options. These are:
:typhoeus
/ :hydra
:parallel
:cache
Typhoeus is used to make fast, parallel requests to external URLs. You can pass in any of Typhoeus' options for the external link checks with the options namespace of :typhoeus
. For example:
HTMLProofer.new("out/", {extensions: [".htm"], typhoeus: { verbose: true, ssl_verifyhost: 2 } })
This sets HTMLProofer
's extensions to use .htm, gives Typhoeus a configuration for it to be verbose, and use specific SSL settings. Check the Typhoeus documentation for more information on what options it can receive.
You can similarly pass in a :hydra
option with a hash configuration for Hydra.
The default value is:
{
typhoeus:
{
followlocation: true,
connecttimeout: 10,
timeout: 30
},
hydra: { max_concurrency: 50 }
}
On the CLI, you can provide the --typhoeus
or hydra
arguments to set the configurations. This is parsed using JSON.parse
and mapped on top of the default configuration values so that they can be overridden.
before-request
callbackYou can provide a block to set some logic before an external link is checked. For example, say you want to provide an authentication token every time a GitHub URL is checked. You can do that like this:
proofer = HTMLProofer.check_directory(item, opts)
proofer.before_request do |request|
request.options[:headers]['Authorization'] = "Bearer <TOKEN>" if request.base_url == "https://github.com"
end
proofer.run
The Authorization
header is being set if and only if the base_url
is https://github.com
, and it is excluded for all other URLs.
Parallel is used to speed internal file checks. You can pass in any of its options with the options namespace :parallel
. For example:
HTMLProofer.check_directories(["out/"], {extension: ".htm", parallel: { in_processes: 3} })
In this example, in_processes: 3
is passed into Parallel as a configuration option.
Pass in parallel: { enable: false }
to disable parallel runs.
On the CLI, you can provide the --parallel
argument to set the configuration. This is parsed using JSON.parse
and mapped on top of the default configuration values so that they can be overridden.
Checking external URLs can slow your tests down. If you'd like to speed that up, you can enable caching for your external and internal links. Caching simply means to skip link checking for links that are valid for a certain period of time.
You can enable caching for this by passing in the configuration option :cache
, with a hash containing a single key, :timeframe
. :timeframe
defines the length of time the cache will be used before the link is checked again. The format of :timeframe
is a hash containing two keys, external
and internal
. Each of these contains a number followed by a letter indicating the length of time:
M
means monthsw
means weeksd
means daysh
means hoursFor example, passing the following options means "recheck external links older than thirty days":
{ cache: { timeframe: { external: '30d' } } }
And the following options means "recheck internal links older than two weeks":
{ cache: { timeframe: { internal: '2w' } } }
Naturally, to support both internal and external link caching, both keys would need to be provided. The following checks external links every two weeks, but internal links only once a week:
{ cache: { timeframe: { external: '2w', internal: '1w' } } }
You can change the filename or the directory where the cache file is kept by also providing the storage_dir
key:
{ cache: { cache_file: 'stay_cachey.json', storage_dir: '/tmp/html-proofer-cache-money' } }
Links that were failures are kept in the cache and always rechecked. If they pass, the cache is updated to note the new timestamp.
The cache operates on external links only.
If caching is enabled, HTMLProofer writes to a log file called tmp/.htmlproofer/cache.log. You should probably ignore this folder in your version control system.
On the CLI, you can provide the --cache
argument to set the configuration. This is parsed using JSON.parse
and mapped on top of the default configuration values so that they can be overridden.
Enable caching in your continuous integration process. It will make your builds faster.
In GitHub Actions:
Add this step to your build workflow before HTMLProofer is run:
- name: Cache HTMLProofer
id: cache-htmlproofer
uses: actions/cache@v2
with:
path: tmp/.htmlproofer
key: ${{ runner.os }}-htmlproofer
Also make sure that your later step which runs HTMLProofer will not return a failed shell status. You can try something like html-proof ... || true
. Because a failed step in GitHub Actions will skip all later steps.
In Travis:
If you want to enable caching with Travis CI, be sure to add these lines into your .travis.yml file:
cache:
directories:
- $TRAVIS_BUILD_DIR/tmp/.htmlproofer
For more information on using HTML-Proofer with Travis CI, see this wiki page.
HTML-Proofer can be as noisy or as quiet as you'd like. If you set the :log_level
option, you can better define the level of logging.
Want to write your own test? Sure, that's possible!
Just create a class that inherits from HTMLProofer::Check
. This subclass must define one method called run
. This is called on your content, and is responsible for performing the validation on whatever elements you like. When you catch a broken issue, call add_failure(message, line: line, content: content)
to explain the error. line
refers to the line numbers, and content
is the node content of the broken element.
If you're working with the element's attributes (as most checks do), you'll also want to call create_element(node)
as part of your suite. This constructs an object that contains all the attributes of the HTML element you're iterating on.
Here's an example custom test demonstrating these concepts. It reports mailto
links that point to [email protected]
:
class MailToOctocat < ::HTMLProofer::Check
def mailto_octocat?
@link.url.raw_attribute == 'mailto:[email protected]'
end
def run
@html.css('a').each do |node|
@link = create_element(node)
next if @link.ignore?
return add_failure("Don't email the Octocat directly!", line: @link.line) if mailto_octocat?
end
end
end
Don't forget to include this new check in HTMLProofer's options, for example:
# removes default checks and just runs this one
HTMLProofer.check_directories(["out/"], {checks: ['MailToOctocat']})
See our list of third-party custom classes and add your own to this list.