Friday, August 31, 2018

Luminous ceph install rough notes

https://gist.github.com/vanduc95/a17f80d0636badd9aa002f2b493b777b
https://www.server-world.info/en/note?os=Ubuntu_16.04&p=ceph&f=1

updated to ubuntu18, then followed the above blogs

Journal lv
root@cephstor1:~# lvcreate -L +8G -n jn_lv "cephstor1_vg"

-----

root@cephstor1:~# ceph-volume lvm prepare --filestore --data cephstor1_vg/cephstor1_lv --journal cephstor1_vg/jn_lv

Running command: ceph-authtool --gen-print-key
Running command: ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring -i - osd new 6cde27f7-e940-4f0e-a053-faae20f3b909
Running command: ceph-authtool --gen-print-key
Running command: mkfs -t xfs -f -i size=2048 /dev/cephstor1_vg/cephstor1_lv
stdout: meta-data=/dev/cephstor1_vg/cephstor1_lv isize=2048 agcount=4, agsize=52428800 blks
= sectsz=512 attr=2, projid32bit=1
= crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0
data = bsize=4096 blocks=209715200, imaxpct=25
= sunit=0 swidth=0 blks
naming =version 2 bsize=4096 ascii-ci=0 ftype=1
log =internal log bsize=4096 blocks=102400, version=2
= sectsz=512 sunit=0 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0
Running command: mount -t xfs -o rw,noatime,inode64 /dev/cephstor1_vg/cephstor1_lv /var/lib/ceph/osd/ceph-1
Running command: chown -R ceph:ceph /dev/dm-1
Running command: ln -s /dev/cephstor1_vg/jn_lv /var/lib/ceph/osd/ceph-1/journal
Running command: ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring mon getmap -o /var/lib/ceph/osd/ceph-1/activate.monmap
stderr: got monmap epoch 2
Running command: chown -R ceph:ceph /dev/dm-1
Running command: chown -R ceph:ceph /var/lib/ceph/osd/ceph-1/
Running command: ceph-osd --cluster ceph --osd-objectstore filestore --mkfs -i 1 --monmap /var/lib/ceph/osd/ceph-1/activate.monmap --osd-data /var/lib/ceph/osd/ceph-1/ --osd-journal /var/lib/ceph/osd/ceph-1/journal --osd-uuid 6cde27f7-e940-4f0e-a053-faae20f3b909 --setuser ceph --setgroup ceph
stderr: 2018-08-31 21:52:02.096100 b759f3c0 -1 journal check: ondisk fsid 00000000-0000-0000-0000-000000000000 doesn't match expected 6cde27f7-e940-4f0e-a053-faae20f3b909, invalid (someone else's?) journal
stderr: 2018-08-31 21:52:02.475694 b759f3c0 -1 journal do_read_entry(4096): bad header magic
2018-08-31 21:52:02.475705 b759f3c0 -1 journal do_read_entry(4096): bad header magic
stderr: 2018-08-31 21:52:02.476242 b759f3c0 -1 read_settings error reading settings: (2) No such file or directory
stderr: 2018-08-31 21:52:02.811785 b759f3c0 -1 created object store /var/lib/ceph/osd/ceph-1/ for osd.1 fsid a509ce31-e9cb-4491-a4d0-bea156feff5e
Running command: ceph-authtool /var/lib/ceph/osd/ceph-1/keyring --create-keyring --name osd.1 --add-key AQAea4lbf/ywMhAA/yGa53/W5IpeH/C1+s/dLw==
stdout: creating /var/lib/ceph/osd/ceph-1/keyring
added entity osd.1 auth auth(auid = 18446744073709551615 key=AQAea4lbf/ywMhAA/yGa53/W5IpeH/C1+s/dLw== with 0 caps)
Running command: chown -R ceph:ceph /var/lib/ceph/osd/ceph-1/keyring
--> ceph-volume lvm prepare successful for: cephstor1_vg/cephstor1_lv

------

ROOT@CEPHSTOR1:~# ceph-volume lvm activate --filestore

Running command: mount -v /dev/cephstor1_vg/cephstor1_lv /var/lib/ceph/osd/ceph-2
stdout: mount: /dev/mapper/cephstor1_vg-cephstor1_lv mounted on /var/lib/ceph/osd/ceph-2.
Running command: ln -snf /dev/cephstor1_vg/jn_lv /var/lib/ceph/osd/ceph-2/journal
Running command: chown -R ceph:ceph /dev/dm-1
Running command: systemctl enable ceph-volume@lvm-2-25bb6df4-350b-42d7-ae0b-4c9509c8924f
stderr: Created symlink /etc/systemd/system/multi-user.target.wants/ceph-volume@lvm-2-25bb6df4-350b-42d7-ae0b-4c9509c8924f.service → /lib/systemd/system/ceph-volume@.service.
Running command: systemctl start ceph-osd@2
--> ceph-volume lvm activate successful for osd ID: 2
--> ceph-volume lvm activate successful for osd ID: None

--------------
Creating OSDs (NA)
The create subcommand wraps the two-step process to deploy a new OSD by calling the prepare subcommand and then calling the activate
subcommand into a single subcommand. The reason to use prepare and then activate separately is to gradually introduce new OSDs into a
storage cluster, and avoiding large amounts of data being rebalanced. There is nothing different to the process except the OSD will become
up and in immediately after completion.

Do the following step on an OSD node, and as the root user:

ceph-volume lvm create --filestore --data $VG_NAME/$LV_NAME --journal $JOURNAL_DEVICE
For example:

# ceph-volume lvm create --filestore --data example_vg/data_lv --journal example_vg/journal_lv

=================================

root@cephstor1:~# ceph-volume lvm list

====== osd.2 =======

[data] /dev/cephstor1_vg/cephstor1_lv

data device /dev/cephstor1_vg/cephstor1_lv
journal uuid mSMVWf-H2ln-62cy-HeL4-Z1L6-NbNl-a27seu
osd id 2
cluster fsid a509ce31-e9cb-4491-a4d0-bea156feff5e
cluster name ceph
osd fsid 25bb6df4-350b-42d7-ae0b-4c9509c8924f
encrypted 0
data uuid Zmp54b-QPf2-9Bv3-sSug-Trpl-d7eN-7mo1mS
cephx lockbox secret
type data
block uuid Zmp54b-QPf2-9Bv3-sSug-Trpl-d7eN-7mo1mS
block device /dev/cephstor1_vg/cephstor1_lv
crush device class None
journal device /dev/cephstor1_vg/jn_lv

[journal] /dev/cephstor1_vg/jn_lv

type journal
journal uuid mSMVWf-H2ln-62cy-HeL4-Z1L6-NbNl-a27seu
osd id 2
cluster fsid a509ce31-e9cb-4491-a4d0-bea156feff5e
cluster name ceph
osd fsid 25bb6df4-350b-42d7-ae0b-4c9509c8924f
encrypted 0
data uuid Zmp54b-QPf2-9Bv3-sSug-Trpl-d7eN-7mo1mS
cephx lockbox secret
crush device class None
data device /dev/cephstor1_vg/cephstor1_lv
journal device /dev/cephstor1_vg/jn_lv
root@cephstor1:~#

---

root@cephstor2:~# ceph-volume lvm prepare --filestore --data cephstor2_vg/cephstor2_lv --journal cephstor2_vg/jn2_lv
Running command: ceph-authtool --gen-print-key
Running command: ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring -i - osd new d2e93b35-6ddf-4b4d-9d9a-b33063232462
Running command: ceph-authtool --gen-print-key
Running command: mkfs -t xfs -f -i size=2048 /dev/cephstor2_vg/cephstor2_lv
stdout: meta-data=/dev/cephstor2_vg/cephstor2_lv isize=2048 agcount=4, agsize=52428800 blks
= sectsz=512 attr=2, projid32bit=1
= crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0
data = bsize=4096 blocks=209715200, imaxpct=25
= sunit=0 swidth=0 blks
naming =version 2 bsize=4096 ascii-ci=0 ftype=1
log =internal log bsize=4096 blocks=102400, version=2
= sectsz=512 sunit=0 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0
Running command: mount -t xfs -o rw,noatime,inode64 /dev/cephstor2_vg/cephstor2_lv /var/lib/ceph/osd/ceph-2
Running command: chown -R ceph:ceph /dev/dm-1
Running command: ln -s /dev/cephstor2_vg/jn2_lv /var/lib/ceph/osd/ceph-2/journal
Running command: ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring mon getmap -o /var/lib/ceph/osd/ceph-2/activate.monmap
stderr: got monmap epoch 2
Running command: chown -R ceph:ceph /dev/dm-1
Running command: chown -R ceph:ceph /var/lib/ceph/osd/ceph-2/
Running command: ceph-osd --cluster ceph --osd-objectstore filestore --mkfs -i 2 --monmap /var/lib/ceph/osd/ceph-2/activate.monmap --osd-data /var/lib/ceph/osd/ceph-2/ --osd-journal /var/lib/ceph/osd/ceph-2/journal --osd-uuid d2e93b35-6ddf-4b4d-9d9a-b33063232462 --setuser ceph --setgroup ceph
stderr: 2018-08-31 23:24:03.987243 b75ec3c0 -1 journal check: ondisk fsid 00000000-0000-0000-0000-000000000000 doesn't match expected d2e93b35-6ddf-4b4d-9d9a-b33063232462, invalid (someone else's?) journal
stderr: 2018-08-31 23:24:04.644125 b75ec3c0 -1 journal do_read_entry(4096): bad header magic
2018-08-31 23:24:04.644138 b75ec3c0 -1 journal do_read_entry(4096): bad header magic
stderr: 2018-08-31 23:24:04.644711 b75ec3c0 -1 read_settings error reading settings: (2) No such file or directory
stderr: 2018-08-31 23:24:04.987278 b75ec3c0 -1 created object store /var/lib/ceph/osd/ceph-2/ for osd.2 fsid a509ce31-e9cb-4491-a4d0-bea156feff5e
Running command: ceph-authtool /var/lib/ceph/osd/ceph-2/keyring --create-keyring --name osd.2 --add-key AQCygIlbf6yuMxAA21ihuGG1M/mqeZcY+cBaZA==
stdout: creating /var/lib/ceph/osd/ceph-2/keyring
added entity osd.2 auth auth(auid = 18446744073709551615 key=AQCygIlbf6yuMxAA21ihuGG1M/mqeZcY+cBaZA== with 0 caps)
Running command: chown -R ceph:ceph /var/lib/ceph/osd/ceph-2/keyring
--> ceph-volume lvm prepare successful for: cephstor2_vg/cephstor2_lv

---
root@cephstor2:~# ceph-volume lvm activate --filestore
Running command: ln -snf /dev/cephstor2_vg/jn2_lv /var/lib/ceph/osd/ceph-2/journal
Running command: chown -R ceph:ceph /dev/dm-1
Running command: systemctl enable ceph-volume@lvm-2-d2e93b35-6ddf-4b4d-9d9a-b33063232462
stderr: Created symlink /etc/systemd/system/multi-user.target.wants/ceph-volume@lvm-2-d2e93b35-6ddf-4b4d-9d9a-b33063232462.service → /lib/systemd/system/ceph-volume@.service.
Running command: systemctl start ceph-osd@2
--> ceph-volume lvm activate successful for osd ID: 2
--> ceph-volume lvm activate successful for osd ID: None

---

root@cephstor3:~# ceph-volume lvm prepare --filestore --data cephstor3_vg/cephstor3_lv --journal cephstor3_vg/jn3_lv
Running command: ceph-authtool --gen-print-key
Running command: ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring -i - osd new 0f7ab8b6-1c22-4e3c-a46c-3890e8e0d9fc
Running command: ceph-authtool --gen-print-key
Running command: mkfs -t xfs -f -i size=2048 /dev/cephstor3_vg/cephstor3_lv
stdout: meta-data=/dev/cephstor3_vg/cephstor3_lv isize=2048 agcount=4, agsize=52428800 blks
= sectsz=512 attr=2, projid32bit=1
= crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0
data = bsize=4096 blocks=209715200, imaxpct=25
= sunit=0 swidth=0 blks
naming =version 2 bsize=4096 ascii-ci=0 ftype=1
log =internal log bsize=4096 blocks=102400, version=2
= sectsz=512 sunit=0 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0
Running command: mount -t xfs -o rw,noatime,inode64 /dev/cephstor3_vg/cephstor3_lv /var/lib/ceph/osd/ceph-3
Running command: chown -R ceph:ceph /dev/dm-1
Running command: ln -s /dev/cephstor3_vg/jn3_lv /var/lib/ceph/osd/ceph-3/journal
Running command: ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring mon getmap -o /var/lib/ceph/osd/ceph-3/activate.monmap
stderr: got monmap epoch 2
Running command: chown -R ceph:ceph /dev/dm-1
Running command: chown -R ceph:ceph /var/lib/ceph/osd/ceph-3/
Running command: ceph-osd --cluster ceph --osd-objectstore filestore --mkfs -i 3 --monmap /var/lib/ceph/osd/ceph-3/activate.monmap --osd-data /var/lib/ceph/osd/ceph-3/ --osd-journal /var/lib/ceph/osd/ceph-3/journal --osd-uuid 0f7ab8b6-1c22-4e3c-a46c-3890e8e0d9fc --setuser ceph --setgroup ceph
stderr: 2018-08-31 23:35:30.829448 b6e523c0 -1 journal check: ondisk fsid 00000000-0000-0000-0000-000000000000 doesn't match expected 0f7ab8b6-1c22-4e3c-a46c-3890e8e0d9fc, invalid (someone else's?) journal
stderr: 2018-08-31 23:35:31.255897 b6e523c0 -1 journal do_read_entry(4096): bad header magic
2018-08-31 23:35:31.255907 b6e523c0 -1 journal do_read_entry(4096): bad header magic
stderr: 2018-08-31 23:35:31.256244 b6e523c0 -1 read_settings error reading settings: (2) No such file or directory
stderr: 2018-08-31 23:35:31.748767 b6e523c0 -1 created object store /var/lib/ceph/osd/ceph-3/ for osd.3 fsid a509ce31-e9cb-4491-a4d0-bea156feff5e
Running command: ceph-authtool /var/lib/ceph/osd/ceph-3/keyring --create-keyring --name osd.3 --add-key AQBhg4lbMsiHNxAAEylG3lRW3JYFbTPCGmrCUw==
stdout: creating /var/lib/ceph/osd/ceph-3/keyring
added entity osd.3 auth auth(auid = 18446744073709551615 key=AQBhg4lbMsiHNxAAEylG3lRW3JYFbTPCGmrCUw== with 0 caps)
Running command: chown -R ceph:ceph /var/lib/ceph/osd/ceph-3/keyring
--> ceph-volume lvm prepare successful for: cephstor3_vg/cephstor3_lv

----

root@cephstor3:~# ceph-volume lvm activate --filestore
Running command: ln -snf /dev/cephstor3_vg/jn3_lv /var/lib/ceph/osd/ceph-3/journal
Running command: chown -R ceph:ceph /dev/dm-1
Running command: systemctl enable ceph-volume@lvm-3-0f7ab8b6-1c22-4e3c-a46c-3890e8e0d9fc
stderr: Created symlink /etc/systemd/system/multi-user.target.wants/ceph-volume@lvm-3-0f7ab8b6-1c22-4e3c-a46c-3890e8e0d9fc.service → /lib/systemd/system/ceph-volume@.service.
Running command: systemctl start ceph-osd@3
--> ceph-volume lvm activate successful for osd ID: 3
--> ceph-volume lvm activate successful for osd ID: None

-------------
https://stackoverflow.com/questions/45012905/removing-pool-mon-allow-pool-delete-config-option-to-true-before-you-can-destro

root@kvm-01:~# ceph -n mon.0 --show-config | grep mon_allow_pool_delete
mon_allow_pool_delete = true
root@kvm-01:~# ceph -n mon.1 --show-config | grep mon_allow_pool_delete
mon_allow_pool_delete = true
root@kvm-01:~# ceph -n mon.2 --show-config | grep mon_allow_pool_delete
mon_allow_pool_delete = true

root@kvm-01:~# cat /etc/ceph/ceph.conf
[global]
auth client required = cephx
auth cluster required = cephx
auth service required = cephx
cluster network = 10.0.0.0/24
filestore xattr use omap = true
fsid = 41fa3ff6-e751-4ebf-8a76-3f4a445823d2
keyring = /etc/pve/priv/$cluster.$name.keyring
osd journal size = 5120
osd pool default min size = 1
public network = 10.0.0.0/24
[osd]
keyring = /var/lib/ceph/osd/ceph-$id/keyring
[mon.0]
host = kvm-01
mon addr = 10.0.0.1:6789
mon allow pool delete = true

[mon.2]
host = kvm-03
mon addr = 10.0.0.3:6789
mon allow pool delete = true
[mon.1]
host = kvm-02
mon addr = 10.0.0.2:6789
mon allow pool delete = true
----
ceph tell mon.\* injectargs '--mon-allow-pool-delete=true'

requires a reboot

https://swamireddy.wordpress.com/2016/01/26/ceph-diff-between-erasure-and-replicated-pool-type/

Tuesday, August 28, 2018

Linuxise MACBOOK - Ubuntu 16, 18 on MACBOOK

How to Install Ubuntu 16.04 on a Retina MacBook Pro

Getting it running, and getting it running well

Apple laptops make very little sense on paper. Why would I want to spend the equivalent of a down-payment for a car, to get a mid-spec machine with supremely inferior customisability? For years, I’d scoffed at the mysterious, pretentious, eat-what-the-chef-gives-you-or-leave-the-restaurant cult of Apple.

Well, I’m old now. I’ve still got the spoons to lug around a 2.5kg ThinkPad, but I’ve lost a little patience for carrying it on my back through my commute every day. It’s also easier to get stuck in work when you can just bring your laptop places without worrying about how you’re going to tether yourself to the power outlet.

So couple of days ago, I finally bit the bullet and bought myself a MacBook Pro. And I am continually bewildered with how much I fucking hate OS X.

Don’t get me wrong — OS X is great for a lot of people. It’s specifically designed to squeeze every bit of performance out of Apple’s hardware, and it’s simple enough to let people get things done. Don’t fuck with your operating system; everything has been meticulously tuned for your pleasure. This is sensible in every way. It also fills me with uncontrollable resentment.

Aside from the initial discomfort of getting used to a new operating system, the Window Management, iCloud integration and miscellaneous features designed for end-users also clashed horribly with my existing workflows and concerns as a pentester. I need a certain level of control over my computing resources, and I am much more familiar and comfortable with the security models for more popular Linux distributions. This is a guide to do an oddly-specific thing, for people with oddly-specific needs. Why would you want to single-boot Linux on a perfectly good MacBook Pro with a great operating system designed to make the most out of the laptop’s hardware? Maybe you don’t want sleepness nights wondering about what the hell goes on in `brew cask`. Maybe you want to breathe new life into old(er) hardware, or you want to avoid your machine going out to pasture by the phenomenon Jamie has (bewilderingly) coined apple-planned-obsolescence-daemon.EXE. Maybe your personal workflow is just more better on Linux.

I’m certainly not the one judging you as I gently stroke the belly of my UbuntBook Pro to check that it’s running at a reasonable temperature.

Expectations and Caveats

It’s probably a bit optimistic to expect that you’re going to get anywhere near the battery life with a Linux install as with the native OS X operating system. My machine is second-hand, and I suspect it has had a battery replacement (84 battery cycles in a gunky 2014 machine seems slightly odd). With the modifications listed, I get about 5–6 hours of battery life under moderate loads, including running virtual machines and playing loud music. Even though this is better than any other laptop I have owned, this is a far cry from the 8–10 hours Apple laptops are famous for.

This writeup also includes some experimental drivers for various MacBook components. While these ported drivers have been open-sourced, and I am happy that they work on this setup, installing these should be done at your own risk! They are unsupported, and vetting source is hard. Installing any non-repository code requires a certain level of caution, as non-repudation of the source is trickier and there are fewer guarantees that someone won’t piss upstream in your PPA and deliver some exotic ransomware to your desktop.

Speaking of doing things at your own risk — that goes for pretty much everything else that’s about to be explained here. This tutorial assumes basic Debian Linux knowledge. Note that these instructions have only been tested with vanilla Ubuntu 16.04 LTS (4.4.0–21.37 kernel) on a 15" MacBook Pro 11,2 without a GPU. Do not mail me your feces if following these instructions bricks your machine.

Installation

Getting Started

First, you’ll need to create a bootable USB stick for your Ubuntu install.

The tool I used, and one I strongly recommend, is Mac Linux USB Loader. It automatically creates a UEFI and Mac-compatible, bootable Linux disk with minimal hassle. It also means that you don’t have to jump through the hoops of using rEFIt or rEFInd to get your image to boot in an EFI-compatible manner. There’s also a couple of neat features in a MLUB boot disk that can make your installation process quick and successful, such as a UEFI bootloader with options on how your booted kernel treats the underlying filesystem(s) on the booting machine.

I created the boot disk using MBUL and a verified image of 16.04 (Xenial Xerus) downloaded from the official Ubuntu site.

Next, rebooting and pressing the Option (alt) key with the USB drive plugged into the computer should bring up a screen with boot options.

Initial screen when booting from a MBUL disk.

Choose the one that says EFI.

When you are asked to select your Enterprise boot options, choose 2 — Modify Linux kernel boot options.

We’re looking to modify the Linux kernel boot options first.

The boot options we’re interested in is 8 — gpt. If you are not planning to erase your entire disk and replace it with your Linux installation, skip this step.

Select 0 after all desired boot options have been highlighted.

A side note is that single-booting Ubuntu on a MacBook has become a lot less painful since it included automatic UEFI boot support for single-boot systems. This, combined with Mac Linux USB Loader, makes the initial installation a gentle and happy affair. You shouldn’t need to do horrible GRUB2 shit post-install — it just works.

I chose ‘Guided — use entire disk and set up encrypted LVM’ during install, which was advised against in some of the older tutorials I read, and everything turned out pretty okay.

Another thing to note here is that I opted to overwrite the unused portions of my disk during install. This can help with things like initial CPU and power consumption, as the journaling daemons will be slowly formatting your drive as you use your freshly-installed OS if you don’t do this during installation. Zeroing during installation was pretty fast on my 256GB SSD, and comparing between a zeroed and non-zeroed install I noticed a small improvement in CPU usage for the journaling daemons.

The Install Wizard is relatively self-explanatory, so I won’t go into too much detail here. It’s important that you select the option to include non-free software during the installation. This is so that you’re able to install the Broadcom Wireless Card drivers and use the WiFi on your laptop. If WiFi doesn’t work out the box, you can run:

$ sudo apt-get install bcmwl-kernel-source curl

 Ensure you have curl for make commands (they download packages)

at a later stage to fetch the drivers manually.

Stemming the Power Haemorrhage

Kworker Bug Workaround

Congrats, you’ve thrown some Linux at your MacBook disk! You’ve probably noticed that your computer is running pretty hot at this point. It’s exerting itself, and that’s slightly concerning. In many more recent Linux kernels, a bug causes kworker threads to chew CPU in the absence of any user-imposed load. The reasons for this are manyfold, but one of them is an ACPI interrupt storm. The following instructions come, without much modification, from this post.

To find the gpes (general purpose events) affected by this issue, run the following:

$ grep . -r /sys/firmware/acpi/interrupts

You should see some output that looks this:

/sys/firmware/acpi/interrupts/sci: 1322202
/sys/firmware/acpi/interrupts/error: 0
/sys/firmware/acpi/interrupts/gpe00: 0 invalid
/sys/firmware/acpi/interrupts/gpe01: 0 invalid
/sys/firmware/acpi/interrupts/gpe02: 0 invalid
/sys/firmware/acpi/interrupts/gpe03: 0 invalid
/sys/firmware/acpi/interrupts/gpe04: 0 invalid
/sys/firmware/acpi/interrupts/gpe05: 0 invalid
/sys/firmware/acpi/interrupts/gpe06: 1321927 enabled
[…]
/sys/firmware/acpi/interrupts/sci_not: 0
/sys/firmware/acpi/interrupts/ff_pmtimer: 0 invalid
/sys/firmware/acpi/interrupts/ff_rt_clk: 0 disabled
/sys/firmware/acpi/interrupts/gpe_all: 1322342

It is necessary to disable the affected gpe lines, ie. those that show unusually high values. Backup the target gpe file descriptors first:

$ sudo cp /sys/firmware/acpi/interrupts/gpe[XX] /path/to/backup

Where [XX] denotes the number of the offending gpe. The command to disable the gpe line is:

$ sudo echo "disable" > /sys/firmware/acpi/interrupts/gpe[XX]

To persist this, open crontab:
$ sudo crontab -e

Add the following line:

@reboot echo "disable" > /sys/firmware/acpi/interrupts/gpe[XX]

Save and exit. This workaround does not work after suspend, so configure it to do this with the following:

$ sudo touch /lib/systemd/system-sleep/disable_gpe[XX]  
$ sudo chmod +x /lib/systemd/system-sleep/disable_gpe[XX]  
$ sudo vi /lib/systemd/system-sleep/disable_gpe[XX]

If you don’t like vi, substitute it with the text editor of your choice. In the file, paste the following code — again, replacing the [XX] with the number of the interrupt line to be disabled:

#!/bin/bash  
 case "$1" in  
   thaw|resume)  
     echo disable > /sys/firmware/acpi/interrupts/gpe[XX] 2>/dev/null  
     ;;  
   *)  
     ;;  
 esac  
 exit $?

Save, and exit. You can repeat these steps for each interrupt line that you see a high value for during the diagnostic phase. Note that pm’s sleep.d is no longer the primary agent for running sleep and wakeup scripts — Ubuntu 16.04 uses systemd’s system-sleep instead. I had to disable both gpe06 and gpe17.

Fixing the Marvell PCIe SSD Bug

A bug in the Linux drivers for the Marvell PCIe SSD controller in the MacBook will cause disk performance degradation and unnecessary CPU grind. This occurs when the disk is set to maximum_performance. A script in pm-utils can be altered as a workaround.

Open the pm-utils script:

$ sudo vi /usr/lib/pm-utils/power.d/sata_alpm

And replace the following line:

false) set_sata_alpm max_performance;;

with this one:

false) set_sata_alpm min_power;;

Save and exit. After the above two changes, reboot. You should notice an immediate improvement in the temperature and CPU consumption of your system if you have a Marvell chip on your SSD.

Taking Control of the Hardware

Installing Sensor, Fan and Ambient Light Sensor Controls

The sensor configuration for your MBP will leave a lot to be desired out the box. MacBook with OSX uses a variety of optimisations to make sure that the computer only uses power as necessary while maximising performance and usability. The Mac fan control driver in the repository, macfanctld, will do a passable job, but the goal here is to emulate the behaviour on an OSX MacBook for optimal performance.

First, tell the kernel to load some Apple-specific modules by opening /etc/modules:

$ sudo vi /etc/modules

Add the following entries to the file:

coretemp
applesmc

Save and exit.

Now, it’s time to install the daemons that will make use of these kernel modules.

Dgraziotin’s mbpfan is what we want to be using instead of macfanctld. As explained by @philpl here,

The Fan Control Daemon obviously controls your fan speeds based on the temperature sensors’ readouts. This program is pretty similar to the way OS X handles the fan speeds. Another popular program to do this, called macfanctld just races up the fan speed in bursts. This daemon however exponentially rises the fan speed according to the temperature and lowers it quadratically when the temperature drops.

This is good and cool! You’ll need to compile this from its Git source for now. Set up your build environment like so:

$ sudo apt-get install git build-essential
$ sudo mkdir /usr/local/src
$ sudo chmod -R 755 /usr/local/src
$ sudo chown -R $USER:$USER /usr/local/src

Now fetch the source, compile the code, run the tests and register the systemd service with the following:

$ cd /usr/local/src
$ git clone https://github.com/dgraziotin/mbpfan
$ cd mbpfan
$ make
$ sudo make install
$ sudo make tests
$ sudo cp mbpfan.service /etc/systemd/system/
$ sudo systemctl enable mbpfan.service

You can start the daemon with

$ sudo mbpfan

without rebooting, if you want.

Next, Poliva’s Lightum is a nifty program that automatically adjusts the screen brightness and keyboard backlight based on the readings from the ambient light sensor located just next to the webcam. This results in a comfortable user experience while ensuring the OS doesn’t drain the battery by outputting light it doesn’t need to.

You’ll also need to compile this one from its Git source for now — there’s a custom PPA for it, but it’s a) not verified and b) doesn’t work on 16.04 and c) adding unverified PPAs is weird and gross for the most part.

First, install the necessary dependencies:

$ sudo apt-get install libxss-dev libdbus-glib-1-dev

Now fetch the source from the remote repository, compile and run. The installation process should register the daemon to start automatically by default. Note that the original repository is actually here, but I ran into issues compiling on Ubuntu 16.04 because the libsystemd libraries have been consolidated into a single library and renamed. The following commands use my fork of the project containing a small tweak to the Makefile in order to compile successfully:

$ cd /usr/local/src
$ git clone https://github.com/mracter/lightum
$ cd lightum
$ make
$ sudo make install
$ lightum

Fixing Suspend Kernel Bug

The 4.40 kernel included with Ubuntu 16.04 suffers from (or enjoys every moment of) a bug that’ll prevent the computer from suspending when the lid is closed, even if you set this in Power Management. If you want this behaviour, edit the logind config file:

sudo vi /etc/systemd/logind.conf

Uncomment the line that says:

#HandleLidSwitchDocked=ignore

And replace it with this instead:

HandleLidSwitchDocked=suspend

Suspension on lid close should work after your next reboot.

Webcam Drivers

A sore point for the LinuxBooks of old has been that certain models don’t come with working webcam drivers. There had been drivers for the older, USB-based cameras, but since the switch to PCIe interfaces for the webcam, getting the drivers for these had been significantly trickier. This is technically true for the PCIe-based webcam on the MacBook Pro 11,2, but you can build and install some “experimental” drivers that work reasonably well.

modprobe v4l2-common
modprobe videobuf2-dma-sg
modprobe videobuf2-core
cd /etc/local/src

#wget http://security.ubuntu.com/ubuntu/pool/multiverse/i/isight-firmware-tools/isight-firmware-tools_1.6-2build1_i386.deb

wget http://security.ubuntu.com/ubuntu/pool/multiverse/i/isight-firmware-tools/isight-firmware-tools_1.6-2build1_amd64.deb
dpkg -i isight-firmware-tools_1.6-2build1_amd64.deb

git clone https://github.com/patjak/bcwc_pcie.git
cd bcwc_pcie/firmware
sudo make
sudo make install
cd ..
sudo make
sudo make install
sudo depmod
sudo modprobe -r bdc_pci
sudo modprobe facetimehd
#https://www.ubuntuupdates.org/package/core/bionic/multiverse/base/isight-firmware-tools

UPDATE: This driver will compile for later kernel versions, all the way up to 4.8.0–26! Happy days. You will need to build and reload the kernel module each time you update your kernel.

Configuring Touchpad Gestures

Contrary to the days of old, the touchpad works fine out the box in Ubuntu 16.04. It isn’t as sensitive as it used to be, and moving the cursor around doesn’t make you want to break your laptop in half. However, one can still do better, and add gesture support for two-finger scroll and three-finger tap.

First, open a file in an executable directory to house your synclient commands:

$ sudo vi ~/.config/touchpad_settings

Then, paste the following into the file:

#!/bin/bash
synclient VertEdgeScroll=1
synclient VertTwoFingerScroll=1
synclient PalmDetect=1
synclient TapButton3=2

Save, and exit.
To run this script at startup, simply add its path to your Startup Applications.

As an alternative to the script method, you can also configure synclient in your xorg.conf file. More documentation on the synclient parameters can be found here.

Configuring Function Keys

A pretty annoying phenomenon on a fresh install is that the function keys won’t work as expected — a regular press will activate the special functions (change volume, change brightness) instead of registering as an F12 or F2. You can invert this permanently by doing the following:

$ echo options hid_apple fnmode=2 | sudo tee -a /etc/modprobe.d/hid_apple.conf
$ sudo update-initramfs -u -k all

This should configure the Apple keyboard module to enable regular function keypresses by default.

Making the Retina Screen Work for You

Window Managers for HiDPI Displays

At the moment, the text on your screen is probably super duper tiny. The double-edged sword of High-Resolution Screens strikes again. The thoughts and suggestions in this sections are what I find worked for me despite some areas of chagrin.

After playing around with a few window managers, I settled with Unity, despite fucking hating everything about it when it first replaced GNOME in Ubuntu.

Unity: Makes no pretense of the fact that it will munch your CPU and RAM. Also comes with awful crap like unwanted webapp integration and Unity Shopping Lens. It is said that the horrible spyware that sends your Launcher searches to Canonical has been disabled by default, but it’s still there, like a turd in the laundry. However, it looks very “nice”! Chrome scales correctly out the box. And it has the best HiDPI integration of all the window managers I’ve tested on the RMBP. Very few horse-legged slip-ups with window scaling.

XFCE: Vaguely usable after hours of trying to figure out how the fuck to adjust things for high resolution screens. The main point of pride seems to be the ability to adjust the font scaling, like this. However, this still leaves icons, window decorations and scrollbars very smoll, and some applications won’t get the message! I still have XFCE set up for my Kali VM, and found the following command helpful:

xfconf-query -c xsettings -p /Gtk/IconSizes -s "gtk-menu=32,32:gtk-button=38,38:panel-applications-menu=42,42:panel-directory-menu=38,38:gtk-small-toolbar=32,32:gtk-button=32,32:gtk-large-toolbar=38,38:gtk-menu-images=32,32:gtk-dialog=38,38:panel-tasklist-menu=32,32:panel=32,32"

This will increase the size of your icons, and hopefully make your desktop a happier place. The pixel values can be adjusted to one’s liking.

You can also make your desktop icons more readable by adjusting their spaces and label widths. This is done by editing the ~/.gtkrc-2.0 file, which may be created if it does not yet exist. Make sure it contains the following:

set ~/.gtkrc-2.0

style "xfdesktop-icon-view" {
 XfdesktopIconView::ellipsize-icon-labels = 0
 XfdesktopIconView::cell-text-width-proportion = 3.5
}
widget_class "*XfdesktopIconView*" style "xfdesktop-icon-view"

Cinnamon: Despite its accolades around the web praising its out-the-box HiDPI support, I found the scaling to be inconsistent enough to be a nuisance.

LXDE: This one I did not test. It’s supposedly clean, usable and lightweight, but I’ve found very few resources that deal with how to get it usable in a high resolution environment.

i3, xmonad: These I have not tested.

KDE: Untested, but there seems to be some controls for text scaling.

GNOME: Untested.

Configuring Unity For Your Retina Display

As mentioned earlier, Unity7 has (imo) the best support for high-resolution displays. Navigate to All Settings > Displays. There should be a slider in the dialog for your rMBP screen called “Scale for menu and title bars:”. Set this to between 1 and 2, depending on your preference — I set this to 1.5. As mentioned above, Chrome should scale out the box, but you’ll need to tweak Firefox to get it to scale correctly. Typing about:config in the navigation bar should bring you to a page of variables. Search for the layout.css.devPixelsPerPx variable, and change this to a value between 1 and 2. This is also a scaling factor, so a value greater than 1 will make your components bigger on the screen.
After these settings, your computer should be usable for the most part. There is the occasional instance where applications don’t scale with the GTK+ scaling factor, but I found them to be tolerable.

Extending Your Battery Life

Battery Optimisations

I found that my battery life hovered around 3–4 hours after installing Ubuntu and configuring the drivers and workarounds in the previous sections. At this point, a battery optimiser and a few manual tweaks can extend the battery by 1–2 hours. There are a great many alternative ways to manage your battery and power usage out there. This post only covers two such ways. Note that laptop-mode-tools is a very popular, coarser-grained package for automatic power optimisation. However, it is generally accepted that laptop-mode-tools is a legacy program, and will integrate badly with newer power saving utilities, or straight-up not work on newer kernels.

You can install powertop and tlp like this:

$ sudo apt-get install powertop
$ sudo apt-get install tlp tlp-rdw
$ sudo tlp start

Note that both powertop and tlp execute the settings specified in their configuration files. However, tlp will start automatically at startup after initial activation, whereas powertop will require some manual steps to persist the desired states. The two utilities also exert control over some of the same parameters; however, they work well in conjunction with each other and the overlap does not seem to result in disruptive behaviour.

Open the configuration file for tlp:

$ sudo vi /etc/default/tlp

You can review your settings here, and change them if necessary. Official documentation for tlp parameters can be found here. For reference, this is my tlp config file.

powertop will require an initial configuration phase before it can give suggestions on optimising battery usage. The calibration can take a while, so don’t panic or restart your machine before it is done:

$ sudo powertop --calibrate

powertop will now use these measurements to provide suggestions on the automatic tweaks it can perform for you. You can apply them by running

$ sudo powertop --auto-tune

There seems to be a bug in the most recent version of Ubuntu repository’s powertop (2.8 as of time of writing), that breaks the powertop — auto-tune command. If you see the following message:

Cannot load from file /var/cache/powertop/saved_parameters.powertop

after running — auto-tune, then the following workaround is applicable:

$ sudo ln -s /var/cache/powertop/saved_results.powertop /var/cache/powertop/saved_parameters.powertop

This creates a symlink to your results from the calibrate command, with the name that autotune expects. Auto-tune should work after this.

To run powertop automatically at startup, there have been some suggestions to include the command to run powertop in /etc/rc.local. However, this has a high chance of not working, for some reason. If adding the

powertop --auto-tune

command to rc.local doesn’t work for you, you can add it to systemd:

$ sudo vi /etc/systemd/system/powertop.service

Paste in the following:

[Unit]
Description=Powertop tunings

[Service]
Type=idle
ExecStart=/usr/bin/powertop --auto-tune

[Install]
WantedBy=multi-user.target

Save and exit, and mark the file as executable:

$ sudo chmod +x /etc/systemd/system/powertop.service

Now start the service:

$ sudo systemctl enable powertop
$ sudo systemctl start powertop

Other Improvements

Miscellaneous Tips

Hopefully, this post will help save some time in setting up a UbuntuBook that doesn’t suck! There remains a few other things I did for a leaner Linux experience, such as removing or disabling packages and services I didn’t need. Here’s a quick list of additional tweaks that may help you save a few mW of battery, or make you feel better about using Unity. DO NOT AUTOREMOVE if making use of these tweaks!

Remove zeitgest and the geotracking packages that come installed with Unity with the following command:

$ sudo apt-get remove zeitgeist zeitgeist-core zeitgeist-datahub python-zeitgeist rhythmbox-plugin-zeitgeist geoclue geoclue-ubuntu-geoip geoip-database

Disable tracker by following these instructions from the Ubuntu Wiki:
1. Edit /etc/xdg/autostart/tracker-extract.desktop file with root privileges (sudo vim, gksudo gedit …)
2. Add “Hidden=true” to the end of the file
3. Do the same for tracker-miner-apps.desktop, ttracker-miner-fs.desktop, tracker-miner-user-guides.desktop, tracker-store.desktop.
Disable the CUPs daemon at startup:

$ sudo systemctl disable cups-browsed.service

Remove Vino, a VNC server, if you don’t use VNC.
Reduce compiz CPU load by disabling anti-tearing optimisations:

1. Install the compizconfig-settings-manager package.
2. Run the Compiz Config Manager, and select the OpenGL section
3. De-select the field Sync to VBlank.
4. Reboot or restart Unity.
Disable Bluetooth on startup:

$ sudo systemctl disable bluetooth.service

Disable avahi-daemon.
Disable anacron.
Install thermald.
Remove unity-lens modules:

$ sudo apt-get purge unity-scope-audacious unity-scope-chromiumbookmarks unity-scope-clementine unity-scope-colourlovers unity-scope-devhelp unity-scope-firefoxbookmarks unity-scope-gdrive unity-scope-gmusicbrowser unity-scope-gourmet unity-scope-manpages unity-scope-musicstores unity-scope-musique unity-scope-openclipart unity-scope-texdoc unity-scope-tomboy unity-scope-video-remote unity-scope-virtualbox unity-scope-yelp unity-scope-zotero unity-lens-music unity-lens-photos unity-lens-video

Use Firefox (or another less resource-intensive browser) instead of Chrome

Wrapping Up

By now, if you haven’t already bricked your machine, you should have Ubuntu running reasonably well on your Retina Macbook Pro. There is still some trouble integrating such a system with HiDPI, multi-screen setups at the moment, and a couple of things here and there could still use some smoothing out. There’s nothing quite like Safari on Linux desktops, and heavily-tabbed browsing or Virtual Machines will still make your machine run a little warm. But taking advantage of the stronger Apple support on Linux has never been easier, or less inadvisable. And hopefully, more and more people can find themselves in a position to agree.

Racter

How much stack could a full-stack stack if a full-stack could stack full? Security Engineer at Praekelt.org.

unix