This repository contains the Python Barcelona Association page deployed using Hugo.
curl,tar, andsha256sum. Thebin/installscript needs them to download and verify the Hugo binary.
Clone this repository and run the bin/install script. It downloads the pinned
Hugo extended binary into bin/hugo, checks its SHA256 against the checksums
published with the release, and does nothing if the right version is already
there.
You need curl, tar, and sha256sum (or shasum, which macOS ships). You
do not need Python, Go, or npm.
The version lives in .hugo-version, which is the single source of truth.
To move to a new Hugo release, edit that file and run bin/install again.
The site search is built after Hugo, by Pagefind,
over the pages Hugo wrote. bin/install-pagefind downloads the release
pinned in .pagefind-version the same way, checks its SHA256 and puts it at
bin/pagefind; bin/search public then writes the index to
public/pagefind/. CI does both after every build. bin/serve runs no
Pagefind, so /search/ on the dev server says the search is not available
there: build with bin/hugo --minify -d public && bin/search public and
serve public/ with any static server to try it.
You can bring a hugo hot reload server using the bin/serve script. This will bring a local hot reloading server on http://localhost:1313.
A change goes through a pull request against edition, and a merge deploys
the site. CONTRIBUTING.md has the steps.
To create a new sponsor data file, run hugo new sponsors/my-new-sponsor.md. This will create a pre-filled MarkDown file under content/sponsors with that same name.
The MarkDown file for the sponsor will have a FrontMatter section with the following fields:
id: Unique identifier of this sponsor. Should be a string without spaces, preferrably kebab-case.name: Sponsor's name to be shown in all renderings.logo_image: File name for the sponsor logo, which must be underthemes/pybcn_theme/assets/images/sponsors/. An external URL is not accepted: copy the file into the repository. The template loads it withresources.Get, so a file understatic/is not found, andbin/check-contentreports it as an error.url: URL for the sponsor web page.twitter: URL for the sponsor's twitter account.
After this FrontMatter section, any valid MarkDown will be considered generic content to be shown in detail view.
Finally, add this sponsor to the sponsors list in the corresponding level of the desired sponsors page, like content/sponsors/_index.md, for the main sponsors page; content/pyladies_bcn/sponsors.md, for the PyLadies BCN Sponsors' page; or the specific event, if appropriate.
Note that the logo images are limited to 90% of width and 150px height.
Some of the pages in the sites offer different options to the visitor, like the Sponsor Us page. These pages show blocks of different lists of options with different blocks of descriptions, like what they can get, what they need to do, etc...
These pages can be created easily using the options layout. This layout requires an options list containing different areas, which contain blocks. Each block will have an id, an optional name to be shown, and the descriptions list. The descriptions contain name, id, and the content, which must be MarkDown content.
Probably the best example is the sponsor-us.md, but basically, the following might be a example:
layout: options
option:
- name: Area
blocks:
- name: First block
id: first-block
descriptions:
- name: First question
id: first-question
content: First response
To archive external or old sites we need to keep as they are, the bin/archive script can be used. It requires you to have wget installed, available in your $PATH, and working. It will need you to provide the desired URL, and will grab all the pages in the site, storing a browsable static version of them under static/archives/your-site, and published, on merge, under archives/your-site.
When doing this, please, take care to review the links are working and taking the visitor to expected URLs, since this has not been tested thoroughly and could be tricky.
Once the site was archived, you can create a new link in the navigational menu using the url attribute of the menu entries, like we did for old PyDays, PyData, or HacktoberFest Barcelona.
The steps to make a change and open a pull request, what the pull request checks, and which paths wait for a review from the web team are in CONTRIBUTING.md, which GitHub links from the pull request form. It lives there and not here, so there is one description of the process and not two that drift apart. The two sections below are the detail of the checks it names.
The pages of the archive were written from the Meetup export of both groups,
the X exports of the association, the playlists of our channel, the records
pyvideo keeps of PyData Barcelona 2017, and the frozen copies of the old site
under static/archives/. Those sources, the tools that read them, and the
reports of the import live in a separate repository of this organisation,
pybcn/pybcn-archive, which is private: a copy of a public page is not a
secret, but 950 Meetup comments with the name of whoever wrote them, and the
email addresses, the phone numbers, and the personal profiles that the event
descriptions still carry, add up to something that is. Ask a member of the
board for access.
Nothing in this repository needs it. It is the record of where each page came from, and the insurance policy if a page has to be written again.
bin/check-html-safety scans content/ and fails if it finds raw HTML that
turns a content change into script execution, a redirect, a credential prompt
or a page overlay: <script, <style, <iframe, a srcdoc= attribute,
<object, <embed, <form, <meta, <link, <base, an on...= event
handler, a style= attribute (unless its value is only sizes and margins), and
a javascript:, vbscript: or data: URL.
It reads every file under content/ that Hugo renders, not only markdown:
.org, .adoc, .rst and .pandoc content can carry raw HTML too. Images and
PDFs are skipped. A file with any other extension fails the check until the
script lists it as text or as a binary asset. In each text file it reads the
YAML front matter at any depth, both as raw text and after YAML decodes it, and
the body.
pip install pyyaml
./bin/check-html-safety
The pr-checks workflow (.github/workflows/pr.yml) runs it on every pull
request.
The ALLOWLIST at the top of the script is empty. The Google Form and Google
Calendar embeds of the site go through the google-embed shortcode in
layouts/shortcodes/, which checks the host and which an organizer reviews. To
accept new third party markup, put it in a shortcode under layouts/. If an
entry is unavoidable, it is keyed by file path, pattern and the prefix of the
URL the element loads, so it permits one known embed and not any later one in
the same file, and the file that carries it needs a rule in
.github/CODEOWNERS, or any contributor can edit the allowed snippet.
bin/check-rendered reads the pages Hugo built and fails on dangerous markup
in them: an on...= handler on any element, a javascript:, vbscript: or
data: URL in any attribute a browser loads from or navigates to (entities
decoded, the way a browser does), an inline <script> or one loaded from off
this site, an <iframe>, <object>, <embed> or <base>, a form or a <meta refresh> that sends the visitor off this site, a style attribute or <style>
block outside a small grammar of sizes, margins and grid properties, a <link>,
an image, a media file or a click beacon fetched from off this site, and the
three pieces of markup that html.parser and a browser read differently.
It does what the other two checks cannot. bin/check-content and
bin/check-html-safety read content/, which is what an outside contributor
can change, and that is the right place for them to stop. The worst defect
found on this site was not there: a partial assembled the sponsor logo markup
with printf and emitted it through safeHTML, so a sponsor name reached the
page unescaped while every content file was clean and both content checks
passed. Reading the output catches that class wherever it lives: a template, a
shortcode, a partial, the theme, a Hugo internal template or the minifier.
The check needs a build made with -D. Three draft pages are a canary: a
sponsor and a person whose name closes the attribute it sits in and opens a
link to canary.invalid, plus a photo caption that does the same with a
protocol relative address, listed on content/canary.md through the real
sponsor grid, people grid and photo gallery. The check fails if any of the
three did not reach that page as text, or if the page is missing, so the
fixture cannot disappear without notice. The deploy workflow builds without
-D, so the canary never reaches the live site.
./bin/hugo --minify -D -d public
./bin/check-rendered public
The standard library is enough, no pip install. The pr-checks workflow
runs it in the links job, on the same build lychee checks. ALLOWLIST in
the script names each accepted finding by page, kind and either the prefix of
the URL the element loads or the SHA-256 of the inline text, with a reason:
today the Google embeds, the PayPal form with its button image, the
Eventbrite button of PyDay 2018, and two inline scripts in the frozen
archives. An entry that no longer matches anything is reported as
stale, so it gets deleted.
This is a work in progress. All the design and implementation decisions are detailed in Proposta d'estructura (currently written in catalan). All the tasks are managed through our private Trello board.
There are, at least, three ways of embedding JS code in a page, depending on when this needs to be processed:
- Using the extraJS block
- Using the requiredJS block
- Using the script tag wherever needed
Remember the blocks only work in templates, but not in partials.
Also consider whether using defer attribute is interesting or not, which should be preferred.
Currently, there are two sponsor pages implemented: The PyBCN sponsors and the PyLadies BCN. Both pages use the same template, in themes/pybcn_theme/layouts/_default/sponsors.html.
The template collects all the pages under sponsors content and iterates over the levels defined in the corresponding sponsors page, rendering the level name, if any, and then all the summaries for each sponsor defined in the level's sponsors list.
A page that defines no sponsor_levels gets one level with every sponsor under content/sponsors/, sorted by name, 4 to the line. /sponsors/ is that page. It used to carry the list by hand and the list had drifted: 25 sponsors were on their edition page and not on it. The two association accounts, pybcn and pyladiesbcn, are left out of that automatic list, because an edition page lists them under an Organizers level and they are not sponsors of themselves. The PyLadies BCN page defines its own levels, so it keeps its own list.
The sponsor summary is defined in a separate partial themes/pybcn_theme/layouts/partials/sponsor_summary.html. This partial receives the sponsor page to render the summary from.
About the creation of new sponsors, the themes/pybcn_theme/archetypes/sponsors.md file contains the template Hugo will use when the editor runs the command hugo new sponsors/my-new-sponsor.md.
Every person photo is a square, cut once in the repository rather than on each
build. bin/square-photos does the cutting: it finds the face with OpenCV,
crops the largest square it can centre on that face, and resizes to at most
800px, which is what the 2x variant of a 400px grid slot needs. It never
enlarges, because a 200px photo blown up to 800 is the same picture in four
times the bytes.
pip install opencv-python pillow
bin/square-photos --dry-run # list what would change, change nothing
bin/square-photos # rewrite the photos in place
It is run by hand when a photo is added. The build does not run it and neither does CI, so a contributor without OpenCV can still build the site.
Review the result before committing. Face detection is crude: it misses a face in profile or in shadow, and it finds faces that are not there. When it misses, the crop falls back to the upper middle of the picture, which is where a head usually is. When the crop is wrong, the usual cause is the photo rather than the crop: a full-length shot on a beach stays a full-length shot on a beach, because the tool does not zoom in.
bin/check-content rejects a photo that is not square, one over 1200px, and
one over 1000 KB, and it reports the photos under 400px, which the build has
to enlarge and which look soft in the grid. Replacing the file is the only fix
for those.
There are three pages displaying people: PyBCN Organizers, PyLadiesBCN Organizers and PyLadiesBCN speakers. More can be added to display, for example, collaborators of other events, etc.
/people/ is the index of the whole section, and it is a list and not a
grid: one row per person record, grouped by the first letter of the id, with
the roles, the events, the talks, and a link to every page that credits the
person. It needs no front matter of its own and reads the records and
partials/appearances_index.html, so a new person file gets a row by being
added. The template is themes/pybcn_theme/layouts/people/list.html and the
style is in themes/pybcn_theme/assets/scss/people_index.scss. Each person
file builds a page of its own, /people/<id>/, from
themes/pybcn_theme/layouts/people/single.html; the card of a people grid
links to it.
All pages rely on the same template: themes/pybcn_theme/layouts/_default/people.html. The individual person summary is defined in a separate partial template: themes/pybcn_theme/layouts/partials/people_summary.html, and the style are defined in themes/pybcn_theme/assets/scss/people.scss.
The template iterates over the defined levels, and for each level:
- displays the level name, if 'name' is provided
- displays a summary of all the people listed, limiting the number of people per line to the provided number 'people_per_line' (which must be in [12, 6, 4, 3, 2, 1])
To add new people to the site, there is a Hugo archetype defined in themes/pybcn_theme/archetypes/people.md that can be used by running the command: hugo new people/my-new-person.md.
The photo field names a file under themes/pybcn_theme/assets/images/people/.
A remote photo_url is not supported: it would load from a third party on
every page that lists the person and tell that party who viewed it, and
bin/check-content reports it as an error.
bin/new-meetup <the Meetup URL of a past event> writes
content/events/meetups/<date>-<slug>.md. One command, one GET of the public
event page, no API key and no login: Meetup serves the whole event as JSON in
the __NEXT_DATA__ script tag its own front end reads. The retired REST API
is why the archive needed an export at all; this page is what replaced it.
It writes the fields that are a copy of one field each, which is the same set
and the same order the 268 pages of the archive carry, through the same
function in bin/import-meetups: title, date, year, venue,
attendance, meetup_url, and group. attendance is who said yes on
Meetup, not who came.
It refuses an event that is not PAST, because the registrations are not
final and the archive records what happened, and an event whose title says
PyDay or PyData, because the large editions have their own pages. --force
overrides both. It refuses to overwrite a page that is already there.
--dry-run prints the file instead of writing it.
What it does not write, and what to do about each:
| Not written | Why, and the next step |
|---|---|
| The talks | The announcement says them in prose, which no parser reads. --description prints that prose; bin/prompts/extract-talks.md is the prompt a model reads it with; bin/check-extraction compares every extracted field against the source; bin/import-talks writes the block |
| The description | It is the logistics of signing up, and an archive records what happened and not how to register for something that is over |
| A video | It comes from the YouTube channel after the day, through bin/import-videos, and data/channel_videos.yaml is the inventory bin/check-content reads |
| A person page | A page publishes a named person. A speaker who just spoke is reachable, so ask that person first |
| A venue short name | The script reports a venue string that data/venues.yaml does not name. The short name is a choice somebody makes |
bin/new-meetup --discover answers the other half: which meetups are
missing. It reads the past events page of both groups, which carries the ten
most recent as JSON in the same script tag, names the ones no page records,
and prints the command for each. It writes nothing. Ten is more than a year
of a group that meets monthly; a longer gap than that is what the
pybcn-archive export is for.
So the whole path for the meetup of last Tuesday is: bin/new-meetup --discover to see what is missing, bin/new-meetup URL --description,
extract the talks, bin/check-extraction, bin/import-talks,
bin/check-content, and bin/import-videos once the recording is up.
The monthly event page uses the meetup component to show PyBCN and PyLadies BCN meetup events lists. The template embeds the meetup.js code without defer using the requiredJS block. Then, it invokes the meetup-events partial which processes the list of meetup events.
The requests to Meetup's API are done asynchronously and should replace the spinner item with the correct list of events.
It is possible to add a carousel of photos in any regular page by changing the layout to 'carousel'. This layout requires us to define the number of columns [1-12] and a list of the images and (optional) captions to display:
layout: carousel
carousel_columns: 6
carousel_photos:
- photo: folder/img1.jpg
caption: "A nice and descriptive caption"
- photo: folder/img2.jpg
caption: "Another nice and descriptive caption"
- photo: folder/img3.jpg
caption: "Yet another nice and descriptive caption"
This page is designed for events like PyDay or DjangoGirls, and allows to display the description of the event as well as a carousel with images, its sponsors, organizers, speakers, previous editions and the agenda. It also offers the possibility to add as many free text elements as desired.
It allows for different sections to be added, that are displayed in a floating menu:
- Content: textual content
- carousel_photos: list of photos to appear in a carousel next to (if content_columns and carousel_columns are indicated) or below the main content.
- free_text_sections: list of sections (title, id, content) containing textual content
- events (special component, see description below)
- sponsor_levels
- people_sections (special component, see description below)
- previous_editions
The agenda is based on the code in https://www.liquidlight.co.uk/blog/create-an-event-schedule-with-html-table/ and requires to define the duration of the time spans in which we want to divide the slots (e.g. 30 minutes slots), how many parallel tracks does the event have, a list of the time slots and a list of the events.
Each event in the list must indicate:
- the starting and ending time slots (may be the same if the event only spans for one time slot)
- the track lenght (e.g. 4 if the event has 4 tracks and the element should span across all of them, like the welcome session)-
- the title of the event
Each event in the list can indicate:
- the color of the event in the calendar: red, orange, yellow, green, blue, purple (defaults to orange)
- the location where the event will take place (a room, or a url)
- the topic of the event (e.g. Data Science, Security...)
- the type of the event: talk, workshop, coffee, lunch, photo, group, qa, lightning
language: the language the talk is given in, shown as text (e.g.Spanish)title_lang: set it only when the title itself is not in English, withSpanish,Castellano,Catalan,Catalàor a BCP 47 code such ases. It puts alangattribute on the title, so a screen reader reads it with the right voice.languagenever sets it: many talks given in Spanish have an English titlepython_levelandtopic_level: the experience the attendee needs. Use one ofbeginner,intermediateoradvanced
python_level and topic_level take a token, never markup. The site renders
each token as a badge with a coloured dot and the matching text label, so the
level does not depend on colour alone:
| Token | Label | Dot |
|---|---|---|
beginner |
Beginner | green |
intermediate |
Intermediate | amber |
advanced |
Advanced | red |
An unknown token prints as plain text and logs a build warning. The tokens and
the labels live in themes/pybcn_theme/layouts/partials/level_badge.html.
legend is a list of items, not an HTML string. Each item is one of:
legend:
- type: workshop # any event type, plus "sponsor" for the gold star
label: Hands-on # optional, overrides the default label
- type: talk
- separator: true # a visual group break
- level: beginner
- level: intermediate
- level: advanced
The icons and the default labels come from
themes/pybcn_theme/layouts/partials/event_types.html.
spansDuration: 20
numTracks: 4
eventTimes: [8:00, 8:20, 8:40, 9:00, 9:20, 9:40, 10:00, 10:20, 10:40, 11:00, 11:20, 11:40, 12:00, 12:20, 12:40, 13:00]
events:
- start_time_slot: 8:00
end_time_slot: 8:40
track_length: 4
color: green
title: Arrival and registration
location: Registration Area
type: coffee
- start_time_slot: 9:00
end_time_slot: 9:20
track_length: 4
color: blue
title: Welcome Talk
location: Main Stage
type: group
- start_time_slot: 9:40
end_time_slot: 10:00
track_length: 3
color: yellow
title: An interesting talk with a big audience
location: http://www.youtube.com
level: advanced
topic: Data Science
type: talk
- start_time_slot: 9:40
end_time_slot: 10:20
track_length: 1
color: red
title: Some long talk
location: http://www.youtube.com
level: advanced
topic: Data Science
type: workshop
- ...
The people_sections option allows to add a list of sections displaying people, such as organizers, speakers and mentors. Each section can have subsections (e.g. in organizers, we could have 'main organizers' and 'volunteers'). Example:
people_sections:
- title: Organizers
id: organizers
levels:
- people_per_line: 4
people:
- elisabeth-ortega-carrasco
- lpmayos
- natalia
- nuria
- title: Mentors, speakers & volunteers
id: main_organizers
levels:
- name: Mentors
people_per_line: 4
people:
- esperanza
- eli
- sandra
- alex
- name: Speakers
people_per_line: 4
people:
- alberto
- cristina
- magda
- cris-lopez
- maria
- martina
- violeta
- name: Volunteers
people_per_line: 4
people:
- mireia
- josep
This repository explains itself in the file that carries the decision, not in a folder of documents beside the code. That keeps the reason next to the thing it governs, and it means the reason is in front of whoever is about to change it. The cost is that the reasons are scattered, so this is the index.
| Decision | Where it is argued |
|---|---|
A person file builds a page, /people/<id>/, and /people/ is the list that reaches every one of them |
The [[cascade]] comment in config.toml, themes/pybcn_theme/layouts/people/single.html, themes/pybcn_theme/layouts/people/list.html, and issue #201 |
/people/ is a list and not a second card grid, ordered by letter and not by how often a person appears |
themes/pybcn_theme/layouts/people/list.html |
An appearance line calls a page by appearances_title, not by its title |
themes/pybcn_theme/layouts/partials/appearances_index.html |
| A card of a people grid is a link to the page of the person, not a modal that shows a subset of it | themes/pybcn_theme/layouts/partials/people_summary.html |
| The menu sections are a disclosure, not an ARIA menu | themes/pybcn_theme/layouts/partials/nav.html |
| MIT for the code, CC BY-SA 4.0 for the content, and nothing for the photographs | LICENSING.md |
edition is the source branch and master is the built site |
AGENTS.md |
| Raw HTML in content is gated by three checks rather than banned | CONTRIBUTING.md and bin/check-html-safety |
| Imported text is normalized in code, and the model that extracts it is told nothing about typography | bin/canon.py and bin/prompts/extract-talks.md |
| An extraction is checked against the normalized source, and one recorded run has to fail | bin/check-extraction and tests/extraction/README.md |
| A photo PyBCN retweeted is never imported, because its copyright is somebody else's | bin/import-x-photos and LICENSING.md |
An import stages its result for review and never writes content/ itself |
bin/import-x-photos |
One decision has no file of its own, because it is a decision not to act:
The heavy image blobs stay in the Git history. The repository is many
times the size of the site it builds (git count-objects -vH gives the
figure), and the largest objects in it are photographs of several megabytes
each that were replaced long ago by far smaller files. They are all reachable from edition and none from
master, which the deploy workflow force-pushes as a single orphan commit.
Removing them means git filter-repo and a force-push, which changes every
commit hash from the first touched blob onwards: every clone breaks, every
open pull request has to be rebased, and every link to a commit or to a line
of code goes dead, including the ones in our own issues. GitHub starts warning
at 1 GB. The cost is a one-time clone, not a recurring one, so the answer is
not now. If it is ever done, the moment is an empty pull request queue and a
day when everybody can re-clone together.
AUTHORS names every person who has written or reviewed here, in
the order they arrived, with the years they worked on it, the account they
did it from, and their page on the site. Reviewing counts: a change nobody
reads is not finished, and two of the people in the file have done nothing
else. It is written by bin/authors out of the commits, the pull request
reviews, and the person records, and never by hand: a list of people kept by
hand goes stale the day somebody new lands and nobody notices.
bin/authors --check writes the file again in memory and compares, so a new
contributor, a second account of somebody already here, and a name edited on
a person record are all caught. It runs in CI with the token the job already
has, and says it skipped rather than failing when the API cannot be reached.
A person who would rather appear another way changes it in a pull request.
The people behind the events, the speakers, the organizers, the venues, and
the sponsors, are named on the pages of the site, which stay current on
their own. static/humans.txt points at both.
The code of the site (the templates, the stylesheets, the scripts, the
workflows, and the configuration) is under the MIT License. The
content (the text under content/, the code of conduct, this README, and the
contributing guide) is under
CC BY-SA 4.0. The
photographs of people, the person pages, the sponsor logos, the association's
own logos, the third-party libraries, and the archived sites are not licensed,
each for a reason of its own. LICENSING.md lists every path
and says why. The code of conduct was adapted from the
PyLadies one, which is under CC BY-SA 3.0; that licence allows the adaptation
to move to 4.0, and LICENSING.md says how.