Get graylog running on port 80 instead of 9000

If you are building a single service server (and if you are doing anything with log files the chances are that you are... then you might want to get Graylog working on a convenient port, like 80 rather than the default 9000.

By default ports below 1024 are privileged and normal applications are not allowed to run on them.

If you search the internet there are loads of conversations about how to do it the right way... well that is too complicated for me, so here's a quick and dirty way to get it set up.

Just run the service using the root user rather than the graylog user - yeah I know, all kinds of a bad idea if this is externally facing, but if it's internal then I think it'll be alright.

In Ubuntu:

sudo vim /etc/graylog/server/server.conf

Change the port number for rest_listen_uri and web_listen_uri to 80

sudo vim  /usr/lib/systemd/system/graylog-server.service

Change
User=graylog to User=root
Group=graylog to Group=root

Exit and run
systemctl deamon-reload
to apply the changes you just made

Touchpad settings keep reverting - scroll in the wrong direction

I love my HP Envy, but I hate the way that the touchpad settings keep switching back after a few weeks.

This is due to the latest Synaptics touchpad drivers being loaded and it overwriting the settings. This is particularly noticeable as the scroll direction switched to "reverse"... honestly Synaptics why would you force reverse settings on anyone when it's obviously not the "right" way!

Anyway, enough about my pet peeves... how do you fix it.
Simple!

Open up regedit and go to:
Computer\HKEY_LOCAL_MACHINE\SOFTWARE\Synaptics\SynTP\Install

Change the setting DeleteUserSettingsOnUpgrade from 1 to 0#

Exit out of regedit, and that should be all you need to do. Your settings will not be overwritten when you get a new version of the touchpad driver.

Using redis failover with F5 and Kemp load balancers

I normally use an F5 for all my load balancing needs, but recently found myself needing to use a Kemp.

Setting up an F5 is easy, just create a pool, and set up monitor like so:

Basically this detects which node is the master and sets it as the only node active. Upon a failover, there will be a short outage and a new node will be elected and come online.

The principal is the same for the Kemp LB, but unfortunately you cannot send clear text.

You need to set:

Real Server Check Method: Binary Data

Checked Port: 6379 (or whatever port you are using)

Data to Send: 696e666f0d0a717569740d0a (this is the binary equivalent of the info command)

Reply Pattern: 726f6c653a6d6173746572

Find Match Within: 0

Your redis should now be working with your Kemp

Note that this does not work with Authenticated sessions.

Setting up Redis Sentinel as a service in Ubuntu

There are plenty of excellent documents on how to set up a Redis Master/Slave system, but very little on Sentinel.

Once you have Sentinel running reliably on the commend line, you will want to get Sentinel working as a service.

First, open up a new file:
vim /etc/systemd/system/sentinel.service

In there you probably want something like:

[Unit]
Description=Sentinel for Redis
After=network.target

[Service]
LimitNOFILE=64000
User=redis
Group=redis
ExecStart=/usr/local/bin/redis-sentinel /etc/redis/<your configuration file>.conf --daemonize no

[Install]
WantedBy=multi-user.target

You then need to make sure that the user you are using to run the service (in this case redis) has permissions for both the configuration file 

chown redis:redis /etc/redis/<your configuration file>.conf

and the log files you have set up in your configuration file

chown redis:redis /var/log/redis/<your log file>.log

Test everything works as expected by:

systemctl start sentinel

then 

systemctl status sentinel

You should get something that looks like:

If you do, then everything is good and you can set your service to start automatically with:

systemctl enable sentinel

Configuring Graylog for LDAP and Active Directory

Configuring LDAP and Active Directory authentication for graylog is pretty simple.

Server configuration

Go to http://<graylog_server>/system/authentication/config/legacy-ldap

Tick "Enable LDAP" duh!

Set the server type - in my case it's Active Directory

Server address ldap:// <IP address or FQDN of your domain controller> : 389

If you are using this externally, you really should be using ldaps to  ensure that your authentication between the graylog server and the DC is encrypted... in fact it's best practice to do this as standard

System username: This is the full URL for a user who has permissions to browse the AD. In my case I create a service account user just for this purpose. They don't have any other access and don;t forget to set the account and password to never expire.

CN=LDAP-Authentication-Only,OU=Service-Accounts,OU=Users,DC=uk,DC=company,DC=local

Don't know what the full URL is? Just install Softera LDAP Browser and look at the properties for your user and you can extract the URL.

System password: <The service account password>

Test Server connection.. go on... get a green message before proceeding!

User mapping

Search Base DN: You don't want to search the whole AD, so this allows you to specify the folder that you want to search below. Usually this is your Users OU - again use Softera to find the full URL

OU=Users,DC=uk,DC=company,DC=local

User search pattern: This is used to make sure you only have user objects and search for the samAccount name - basically it means you can log in with your short username e.g. jsmith rather than your full email address. Just use the setting they tell you: 

(&(objectClass=user)(sAMAccountName={0}))

Display Name Attribute: How you want your user to appear, I just used displayName, but you could use cn if you wanted "Firstname Lastname" 

At this stage, you can now go to step 5 and do a login test... hopefully it should all go well, and you can save your changes and anyone in the AD can log on!

If it fails, make sure you are not using a username that is already in the graylog user database... this caught me out and I had to delete the local user before it would work.

Group Mapping

The first time you log in with an AD accoint you will notice that you don't have a search option, this is because Graylog have decided that they want regular users to use streams. 

If you want AD users to be admins (and have search capabilities) you need to set up Group Mappings.

First thing to do is set up a couple of Security Groups in AD. I chose to call them "Graylog Users" and "Graylog Admins"

Group Search Base is very similar to the Search Base DN you just set up.. but in this case point it to the OU that has all your groups below it.

OU=Groups,DC=uk,DC=company,DC=local

Group Search Pattern should be set up to tell Graylog to search for groups with a particular string, in my case I used:

(&(objectClass=group)(cn=Graylog*))

Note that this is CaSe SeNsItIve

Group Name Attribute should be cn

Default User Role defines what permissions everyone is created with when they log on for the first time. I left mine at Reader - basic access as I don't want everyone able to break my system.

You should no save your changes. 

Now it's time to map your Active Directory groups to Graylog roles by going to http://<graylog_server>/system/ldap/groups

Just pick your AD groups and assign the permissions you want for members of that group.

Congratulations you have set it up..now log on with your AD/LDAP user and start playing :)

Disable Shutdown and Suspend start menu on Windows 10 Home

Shamelessly ripped from a Microsoft article:

We have to Disable and Remove Shutdown option using Registry Editor.  Open Registry Editor, go to following path: HKCU\Software\Microsoft\Windows\CurrentVersion\Policies\Explorer

Select Explorer in the left-hand pane, right click in the right-hand pane and select New >DWORD; then name it as NoClose.

Double click on NoClose and change the Hex Value to 1; Click OK and Close Registry Editor.

Restart, you will be able to Disable and Remove Shutdown in Windows 7 Home Premium or lower in this way.

Very slow disk access with VMware ESX and HP Gen8 Microserver

I recently swapped my home ESXi server from an ageing Dell desktop for a nice new HP MicroServer Gen8
I love the new server, with it's enterprise level features, and its ability to just use any old SATA drive.
As a special treat I got an SSD as one of the datastores, as well as an existing regular HDD (aka spinning rust).

However, when I started using VMs in anger I was disappointed by the speed they were performing at... in fact things felt slower than on the desktop!

I had installed ESXi on a USB stick and used the latest build direct from the HP website, so everything must be fine on that end... so I must be imagining it, right... right?

The first thing I did was move my VM over from the HDD to the SDD (which took ages for only 20Gb) and observed that it did not feel significantly quicker.

So I whipped out IOmeter and started to do some benchmarking.

I was seeing only 1.1 MB/s read/write on the HDD, having nothing to compare this to, I shrugged and ran a test on the SSD and expected to get a significant improvement... but no all I got was 1.5MB/s. Even worse, I ran IOmeter against my SAN (an HP Gen8 running unRAID and 4 HDDs) and was seeing 89 MB/s!

After some ninja googling, I came across this article

It appears that the driver for the built in HP disks is faulty in the current version and you need to load up an old version to get your performance back.

So, here's how to fix it:

Copy the v88 driver from here: http://vibsdepot.hp.com/hpq/nov2014/esxi-550-drv-vibs/hpvsa/
(Don't worry if you are running ESX6, this will still work despite saying ESX5.5 in the file name).

• Stop all VMs
• Enable ssh-conection if it is not already turned on
• Copy "scsi-hpvsa-5.5.0-88OEM.550.0.0.1331820.x86_64.vib" to /tmp  (using WinSCP)
• Start ssh-conection (using putty)
• change directory to /tmp
  cd /tmp
• Copy the vib file to /var/log/vmware
  cp scsi-hpvsa-5.5.0-88OEM.550.0.0.1331820.x86_64.vib /var/log/vmware/
• Start maintenanceMode
  esxcli system maintenanceMode set --enable true
• Deinstall the running  scsi-hpvsa driver
  esxcli software vib remove -n scsi-hpvsa -f
  This may take a few minutes to complete...
• Install scsi-hpvsa-5.5.0-88
  esxcli software vib install -v file:scsi-hpvsa-5.5.0-88OEM.550.0.0.1331820.x86_64.vib --force --no-sig-check --maintenance-mode
• Restart ESXi
• Disable maintenance mode
• Start VMs

And what was the result?

HDD now 6.15 MB/s (a 459% increase)

SSD now 55.85 MB/s (a 3623% increase!!!!)

Wow!

UPDATE for VMWare 6.5 Update 1

Upon rebooting with the new (old) driver, my VMware instance did not mount the existing HDDs automatically.

To get round this I ran

esxcfg-volume -l

Scanning for VMFS-3/VMFS-5 host activity (512 bytes/HB, 2048 HBs).
VMFS UUID/label: 57ea5aca-e9e426b3-fcce-6805ca2ee445/HDD1
Can mount: Yes
Can resignature: Yes
Extent name: t10.ATA_____ST1000DM0032D1ER162__________________________________Z4Y3LBLN:1        range: 0 - 953599 (MB)

Scanning for VMFS-3/VMFS-5 host activity (512 bytes/HB, 2048 HBs).
VMFS UUID/label: 57ea59d4-98d844d8-e3c8-6805ca2ee445/SSD1
Can mount: Yes
Can resignature: Yes
Extent name: t10.ATA_____Crucial_CT256MX100SSD1__________________________14510E1BBC87:1 range: 0 - 243967 (MB)

Then took did a

esxcfg-volume -m HDD1
esxcfg-volume -m SSD1

and that fixed the issue :)