Steps to set up a local Phabricator installation on a local Fedora machine, make it look close enough to Wikimedia's production instance, make it have some data, and then create tutorial videos. See phab:T214522 for context.
Install the Phabricator software locally
- First follow the installation guide which basically means:
- sudo dnf install httpd mariadb-server php php-gd php-mbstring php-mysqlnd php-opcache php-pecl-apcu-bc php-pecl-zip php-xml
- sudo dnf install php-process (needed for running the daemons which say "posix PHP extension not installed")
- sudo systemctl restart httpd.service
- sudo systemctl restart mariadb.service
- Open browser and go to http://localhost/ to check if httpd is up
- sudo mkdir /var/www/html/phab/
- cd /var/www/html/phab/
- sudo git clone https://github.com/phacility/libphutil.git
- sudo git clone https://github.com/phacility/arcanist.git
- sudo git clone https://github.com/phacility/phabricator.git
- Now follow the configuration guide:
- On this machine, my user account is called 'ak'.
- sudo vim /etc/httpd/conf/httpd.conf:
<VirtualHost *>
ServerName phab.localhost
DocumentRoot /var/www/html/phab/phabricator/webroot
<Directory "/var/www/html/phab/phabricator/webroot">
Options Indexes FollowSymLinks
AllowOverride All
Require all granted
</Directory>
RewriteEngine on
RewriteRule ^/rsrc/(.*) - [L,QSA]
RewriteRule ^/favicon.ico - [L,QSA]
RewriteRule ^/(.*)$ /index.php?__path__=/$1 [B,L,QSA]
ErrorLog /var/log/httpd/error_log
</VirtualHost>
- sudo mysql
- mysql> CREATE USER 'ak'@'localhost.foo' IDENTIFIED BY '';
- mysql> GRANT ALL PRIVILEGES ON *.* TO 'ak'@'localhost.foo' IDENTIFIED BY '';
- sudo /var/www/html/phab/phabricator/bin/config set mysql.host "localhost"
- sudo /var/www/html/phab/phabricator/bin/config set mysql.user ak
- sudo /var/www/html/phab/phabricator/bin/config set mysql.pass ""
- sudo systemctl restart mariadb.service
- sudo /var/www/html/phab/phabricator/bin/storage upgrade --user "ak" --password "" --force
- sudo /var/www/html/phab/phabricator/bin/config set phabricator.base-uri "http://phab.localhost"
- sudo systemctl restart mariadb.service
- sudo systemctl restart httpd.service
- In /etc/php.ini change post_max_size = 8M to post_max_size = 32M
Set up WMF customizations
- Go to http://phab.localhost/ and log in as admin
-
Implement WMF customizations:
- Copy local.json to the location /var/www/html/phab/phabricator/conf/local/local.json which adds WMF Priority and status values, custom wordmark, enables serious business mode, same color scheme, etc by
- Download custom WMF Phab logo and upload it via http://phab.localhost/file/upload, then open that file in browser to get its actual URL, then copy its internal name into /var/www/html/phab/phabricator/conf/local/local.json for the ui.logo value
- Set http://phab.localhost/config/edit/ui.header-color/ to Blue as I'm too stupid to find out how to do that in local.json
- Set http://phab.localhost/config/edit/maniphest.custom-field-definitions/ to the same as production
- Set http://phab.localhost/config/edit/maniphest.points/ to the same as production (otherwise the data creation script will fail)
- Set up (at least some) WMF code customizations:
- sudo mkdir /var/www/html/phab/libext/
- cd /var/www/html/phab/libext
- sudo git clone https://phabricator.wikimedia.org/diffusion/PHEX/phab-extensions.git misc
- git clone https://phabricator.wikimedia.org/diffusion/PHES/phabricator-security.git security
- sudo /var/www/html/phab/phabricator/bin/config set load-libraries '["/var/www/html/phab/libext/misc","/var/www/html/phab/libext/security/src"]' (this content will get added in the file /var/www/html/phab/phabricator/conf/local/local.json)
- Basically see upstream docs or Mukunda's comments though I didn't fully get it I'm afraid
-
Check that the same Priority values are in place as in WMF Phabricator via http://phab.localhost/config/edit/maniphest.priorities/; e.g. rename "Wishlist" priority field value to "Lowest".
-
Set up some custom forms:
-
Set policy.allow-public to Allow Public Visibility (otherwise we cannot set Visible To in forms to Public)
-
All forms require fiddling with "Edit Form Configuration" and "Change Default Values" and also "Change Field Order".
-
Create basics of the custom Security task form and the custom NDA protected task form by creating them via http://phab.localhost/transactions/editengine/maniphest.task/edit/form/default/ (as these forms are shown as defaults in the WMF Favorites dropdown)
-
In the video, do NOT show using the Security task form or the WMF-NDA form because way too complicated to set up locally with shown project tags and projects. Maybe another time.
-
Make sure that the default task form 2 is NOT marked for editing, so only Form 4 is used for editing. Otherwise we could not change the status of tasks via the dropdown.
-
Go to default form http://phab.localhost/transactions/editengine/maniphest.task/edit/2/ and change the form name from "New Task" to "Create Task".
-
Create a form dedicated for editing. Make that Edit Form 4 show exactly the same fields and field order as in https://phabricator.wikimedia.org/transactions/editengine/maniphest.task/view/3/
-
(See the list of all WMF Forms if you're too bored).
-
Right now there is still no way to change the Priority value in tasks. Hence: Go to http://phab.localhost/transactions/editengine/maniphest.task/ and create a second form. Click "Mark as Edit Form". Do not click "Mark as Create Form". Then go to "Lock/Hide fields" and sync with https://phabricator.wikimedia.org/transactions/editengine/maniphest.task/view/3/, e.g. hide Security field etc.
-
Set up Favorites menu:
-
Remove unused Applications: Go to http://phab.localhost.org/applications and configure->uninstall Phrequent (so you don't see "Start Tracking Time" in task sidebar), Nuance, Phriction, ChatLog, Conpherence (so there won't be an icon in the top bar), Diviner, Drydock, Fund, Guides, Phortune, Phragment, Releeph.
-
UI buttons: Set http://phab.localhost/config/edit/phabricator.serious-business/ to false
-
Set up global sidebar from production to http://phab.localhost/home/menu/configure/global/
-
Set up front dashboard's panels (because needed for dashboard): Copy W727, W6, W3, to http://phab.localhost/dashboard/panel/edit/. Add W3 text panel to W6 tab panel. Ignore the other panels down there - shrug.
-
Set up front dashboard (because needed for sidebar): Copy dashboard from production to http://phab.localhost/dashboard/edit/form/default/, then use Add Dashboard to Menu 🡒 Add to Home Page Menu 🡒 Add to Global Home Menu, after that go to http://phab.localhost/home/menu/configure/global/ and move this dashboard to the very top to make it the default view
Create test data
-
Create user accounts: Create six standard user accounts (as Users can not be created via the API) on http://phab.localhost/people/create/:
Maria / Maria / maria@example.com, Kate / Kate / kate@example.com, Yves / Yves / yves@example.com, Cassidy / Cassidy / cassidy@example.com, Finley / Finley / finley@example.com, and for the sake of faking it also as a standard user gerritbot / Gerrit / gerritbot@example.com.
(See Configuring accounts how to log into these accounts from a console, basically: /var/www/html/phab/phabricator/bin/auth recover someUsername and then use the shown link.)
Log in as Gerritbot, go to its profile, and set the user profile image to https://phab.wmfusercontent.org/file/data/ctemipegegt3vq2bjkqs/PHID-FILE-x4lhwocmrnntw7ph3wwp/profile.
-
DB fiddling: mysql -u ak and then:
-
Get rid of that "Email Not Verified" in front of user names via SQL command: UPDATE phabricator_user.user u SET isEmailVerified = "1" WHERE (u.realName = "Finley" OR u.userName = "gerritbot");
-
Get the PHID-USER-xxxxxxxxxxxxxxxxxxxx of the user named Finley; then fake the "MediaWiki User" link on the user profile: INSERT INTO phabricator_user.user_externalaccount (id, phid, userPHID, accountType, accountDomain, accountID, dateCreated, dateModified, username, emailVerified, accountURI,properties,providerConfigPHID) VALUES ('1', 'PHID-XUSR-abcdefghijklmnopqrst', 'PHID-USER-xxxxxxxxxxxxxxxxxxxx', 'mediawiki', 'mediawiki', '15764393', '1410977885', '1410977885', 'Finley', '0', 'https://www.mediawiki.org:/w/index.php?title=User:Finley', '{}', 'PHID-AUTH-lzggdu7sabdjmwyrzigm');
-
Install jq package; needed for the script to create projects and tasks
-
Start daemons via /var/www/html/phab/phabricator/bin/phd start - needed for search result indexing (and maybe also for creating test data)
-
As of August 2020, edit Phabricator's PHP code to work around https://secure.phabricator.com/T13553 (see test data creation script warning output at beginning)
-
Log in as Maria via /var/www/html/phab/phabricator/bin/auth recover Maria, then go to Settings > Conduit API Tokens, then "Generate Token"
-
Use token from previous step to run shell script at https://gitlab.com/aklapper/phabricator-lorem-ipsum/-/blob/master/phabvideoscriptdata.sh to create projects and tasks (some with assignees and priorities), add folks as members to projects, etc.
-
If things went wrong: After screwing up creating data in the Phab test instance, run the following SQL command to remove data from the maniphest and project databases:
START TRANSACTION; use phabricator_project; TRUNCATE TABLE edge; TRUNCATE TABLE edgedata; TRUNCATE TABLE project; TRUNCATE TABLE project_column; TRUNCATE TABLE project_columnposition; TRUNCATE TABLE project_columntransaction; TRUNCATE TABLE project_customfieldnumericindex; TRUNCATE TABLE project_customfieldstorage; TRUNCATE TABLE project_customfieldstringindex; TRUNCATE TABLE project_datasourcetoken; TRUNCATE TABLE project_project_fdocument; TRUNCATE TABLE project_project_ffield; TRUNCATE TABLE project_project_fngrams; TRUNCATE TABLE project_project_fngrams_common; TRUNCATE TABLE project_slug; TRUNCATE TABLE project_transaction; TRUNCATE TABLE project_trigger; TRUNCATE TABLE project_triggertransaction; TRUNCATE TABLE project_triggerusage; use phabricator_maniphest; TRUNCATE TABLE edge; TRUNCATE TABLE edgedata; TRUNCATE TABLE maniphest_customfieldnumericindex; TRUNCATE TABLE maniphest_customfieldstorage; TRUNCATE TABLE maniphest_customfieldstringindex; TRUNCATE TABLE maniphest_nameindex; TRUNCATE TABLE maniphest_task; TRUNCATE TABLE maniphest_task_fdocument; TRUNCATE TABLE maniphest_task_ffield; TRUNCATE TABLE maniphest_task_fngrams; TRUNCATE TABLE maniphest_task_fngrams_common; TRUNCATE TABLE maniphest_transaction; TRUNCATE TABLE maniphest_transaction_comment; use phabricator_user; TRUNCATE TABLE edge; TRUNCATE TABLE edgedata; TRUNCATE TABLE phabricator_session; COMMIT;
-
Log in as @gerritbot via /var/www/html/phab/phabricator/bin/auth recover Gerritbot and then go to task "Repair leaking pipe in bathroom" and create a fake Gerrit comment:
Change 12345678 had a related patch set uploaded (by Cassidy; owner: Cassidy):
[house/bath@main] Fix memory leak in bathroom pipe (`1a2b3c4de`)
https://gerrit.wikimedia.org/r/12345678
Then also add the project tag #Patch-For-Review. Then log out.
-
Log in as Cassidy, then go to http://phab.localhost/tag/plumbers, click "Workboard" in sidebar, then "Create workboard", then in upper right corner click "Manage > Add Column", then create column "In Progress". Then on the workboard, move the task "Repair leaking pipe in bathroom" into that column. (Because it is not supported to set up workboard columns via API.)
-
Log in as Cassidy, then manually make "Repair leaking pipe in bathroom" a subtask of "Take a bath". (I'm too stupid for the API call to get an ID of an existing task.)
-
Log in as admin, then go to http://phab.localhost/herald/create/, select "Maniphest Tasks", select "Global Rule", set "When all of Title contains grumpy", then every time Add Projects #ACME".
-
Log in as Finley, then go to http://phab.localhost/p/Finley/, click "Manage", click "Edit Profile". Set "Title" to "Chief wizard in training". Set Blurb to
Chief wizard; see [my mediawiki.org user profile](https://www.mediawiki.org/w/index.php?title=User:Finley) for more info.
-
Log in as Finley, then go to "Deploy a grumpy dog" task T6 and add a comment "Do we have any preferences on which kind of dog?" to trigger the Herald rule.
-
Go to "Food-Plants" subproject, create a workboard for it, and then go to "Manage > Edit Menu" and make the Project Details the default view (click the pin).
-
Log in as Finley, then go to "Grow potatoes" task, click "Flag For Later" in side bar, select red flag and add comment "Too yummy to forget!", then set flag (so we have something already in the Flag list)
Record raw audio
- Record audio script (with a better microphone on a silent night) as wav (e.g.: arecord -f cd -d 1200 test.wav)
- Use post-recording noise reduction (Audacity etc.) if needed over audio files
Create raw static video parts
This is about the static intro slide and end slides for each video, plus showing static screenshots of web pages (without any mouse pointer activity)
- Create PNG screenshot files of web pages shown in the vide, in format 1675x980px (see below)
- Create custom slide screenshots as SVG files in Inkscape (File > New from Template > Video) with size 1675x980px (see below), for beginning and end slides of videos
- Convert SVG files to PNG files:
for file in *.svg; do convert ${file} ${file}.png; done
- Convert the PNG files to videos (the -t value is the video length in seconds):
ffmpeg -loop 1 -i slide.png -c:v libvpx-vp9 -b:v 2M -t 7 -strict experimental -shortest videoXX-YY-cropped.webm
Prepare for recording screencast videos
- Phabricator:
- Start daemons via /var/www/html/phab/phabricator/bin/phd start (needed for search result indexing in the screencast videos)
- Log into Phabricator as Finley
- System:
- Under GNOME Settings, go to Universal Access, and set Seeing > Cursor Size to Large, and set Pointing & Clicking > Locate Pointer to On (to show an effect when pressing the Ctrl key).
- Make screencasts in GNOME not stop after 30 seconds:
gsettings set org.gnome.settings-daemon.plugins.media-keys max-screencast-length "uint32 600"
- Make sure to log into Wayland because the pointer stuff plus text highlighting via Ctrl is so broken under X.org
- Browser:
- Sort out width of browser window for screencasts. I wanted 16x9 format but some in-between Firefox update changed the titlebar UI element behavior and height which I did not notice, so I ended up with the strange 1675x980px ratio (instead of 1675x942px which I originally wanted, as 942÷9×16 = 1675), so start browser via firefox --width 1675 and move resulting browser window to upper left corner.
- Increase zoom level on local Phabricator instance and mediawiki.org and phabricator.wikimedia.org in the browser to 150%
Record raw screencast videos
- Record the actual screencast video parts (via GNOME's built-in fullscreen capturing), while listening to the audio files in parallel.
- Save each file as videoXX-YY-raw.webm
- If needed, adjust audio files to screencast video parts, e.g. by adding silence breaks to the audio files
Postprocess (crop, merge, concatenate, export)
- [raw🡒cropped] Resize/crop area of screencast video parts:
for file in videoXX-YY-raw.webm; do ffmpeg -i "$file" -b:v 2M -filter:v "crop=1675:980:0:100" "${file/%-raw.webm/-cropped.webm}"; done
- Make sure each audio part and corresponding video part have pretty much the same length - if there are huge differences, then fix manually.
- You can list the lenghts of all webm and wav files via
for file in video*.w*; do echo "$file"; ffmpeg -i "$file" 2>&1 | grep Duration; done
- [cropped🡒av] Merge each video part and audio part into one file (no reencoding to keep quality of both audio and video until last rendering):
for file in video*-cropped.wav; do ffmpeg -i "${file/%.wav/-cropped.webm}" -i "$file" -c:v copy -c:a opus -strict experimental "${file/%.wav/-av.webm}"; done
- [av🡒rc] Concatenate video parts into each tutorial video without reencoding:
- Create text file videoXX-filelist.txt which must include lines in the format of: file videoXX-YY-av.webm
- ffmpeg -f concat -i videoXX-filelist.txt -c:v copy ./videoXX-rc.webm
- [rc🡒final] Final export: Use VP9 codec via two-pass and convert audio to mono:
- ffmpeg -y -i videoXX-rc.webm -c:v libvpx-vp9 -pass 1 -b:v 1000K -threads 1 -speed 4 -tile-columns 0 -frame-parallel 0 -auto-alt-ref 1 -lag-in-frames 25 -g 9999 -aq-mode 0 -an -f webm /dev/null
ffmpeg -i videoXX-rc.webm -c:v libvpx-vp9 -pass 2 -b:v 1000K -threads 1 -speed 0 -tile-columns 0 -frame-parallel 0 -auto-alt-ref 1 -lag-in-frames 25 -g 9999 -aq-mode 0 -c:a libopus -b:a 64k -ac 1 -f webm videoXX-final.webm
- Afterwards you must check the resulting videos as artefacts can happen, e.g. slides becoming unreadable. Hence sometimes stick with the original -rc file instead of the -final file instead.
Create subtitle files
- (Re)use the audio script file: Split it into one file per video, then add time markers to (srt format) subtitle files (e.g. via gnome-subtitles or Aegisub application). See c:Commons:Timed_Text.
Publish
- Upload resulting videos to Wikimedia Commons via UploadWizard
- Add the following categories: Category:Phabricator_Tutorial_Series (shown in the videos), Category:Videos about Phabricator, and Category:Screencast videos of free software
- License the videos under the CC0 1.0 Universal License.
- Manually edit the License section of each resulting File: page on Wikimedia Commons that Phabricator itself is licensed under the Apache License, Version 2.0.
- On each File: page, click the "TimedText" tab and then paste the content of the corresponding subtitle file
Announce
