Opened 3 years ago

Closed 3 years ago

Last modified 3 years ago

#25778 closed Cleanup/optimization (fixed)

Update docs links to use https

Reported by: Tim Graham Owned by: Anderson Resende
Component: Documentation Version: master
Severity: Normal Keywords:
Cc: jon.dufresne@… Triage Stage: Accepted
Has patch: yes Needs documentation: no
Needs tests: no Patch needs improvement: no
Easy pickings: yes UI/UX: no

Description

Search the docs for links using "http:" and update them to "https:" for all sites that support it.

Change History (5)

comment:1 Changed 3 years ago by Anderson Resende

Owner: changed from nobody to Anderson Resende
Status: newassigned

comment:2 Changed 3 years ago by Jon Dufresne

Cc: jon.dufresne@… added
Has patch: set

comment:3 Changed 3 years ago by Tim Graham <timograham@…>

Resolution: fixed
Status: assignedclosed

In 7aabd62:

Fixed #25778 -- Updated docs links to use https when available.

comment:4 Changed 3 years ago by Tim Graham <timograham@…>

In bf76cf07:

[1.9.x] Fixed #25778 -- Updated docs links to use https when available.

Backport of 7aabd6238028f4bb78d0687bbccc97bcf634e28b from master

comment:5 Changed 3 years ago by Jon Dufresne

For future reference, here is the quick and (extremely) dirty script I used to search and replace these URLs. I also did a manual check of the final diff and had to make some slight corrections. Feel free to use however you find convenient.

#!/usr/bin/env python3

import os
import re
import requests
import subprocess
import urllib

URL_RE = re.compile(br'http://[a-z0-9.]+')
EXCLUDE = [
    '.git',
]


done = set()

for dirpath, dirnames, filenames in os.walk(os.getcwd()):
    j = 0
    for i, dn in enumerate(dirnames):
        if dn in EXCLUDE:
            del dirnames[i - j]
            j += 1

    for fn in filenames:
        if fn.endswith('.mo'):
            continue

        path = os.path.join(dirpath, fn)
        print('%s' % path)
        with open(path, 'rb') as fp:
            for line in fp:
                pos = 0
                # TODO: Ports?
                while True:
                    match = URL_RE.search(line, pos)
                    if not match:
                        break
                    pos = match.end()

                    old = match.group(0).decode('ascii')
                    if old not in done:
                        o1 = urllib.parse.urlparse(old)
                        new = 'https://%s' % o1.netloc
                        print('  checking %s' % new)
                        try:
                            r = requests.head(new, allow_redirects=True, verify=True, timeout=10)
                        except (requests.exceptions.ConnectionError,
                                requests.exceptions.SSLError,
                                requests.exceptions.InvalidURL,
                                requests.exceptions.ReadTimeout):
                            pass
                        else:
                            if r.status_code == 200:
                                o2 = urllib.parse.urlparse(r.url)
                                # Check didn't redirect to http.
                                if o2.scheme == 'https':
                                    print('    %s -> %s' % (old, new))
                                    # Works with https
                                    subprocess.check_call([
                                        'find',
                                        os.getcwd(),
                                        '-name', '.git', '-prune', '-o',
                                        '(',
                                        '-type', 'f',
                                        '-not', '-name', '*.mo',
                                        ')',
                                        '-exec',
                                        'sed',
                                        '-i',
                                        '-e',
                                        's|%s|%s|g' % (old, new),
                                        '{}',
                                        ';'
                                    ])
                    done.add(old)
Note: See TracTickets for help on using tickets.
Back to Top