How to replace an LSI raid disk with MegaCli

Publicat la 20 May 2015 Niciun comentariu

The MegaCli64 command has a lot of command line switches and the syntax is also cryptic. If you have identified a failed, or failing disk, it is possible to replace it using the MegaCli utility. The following commands I found useful when trying to physically identify a failed disk and replace it.

IMPORTANT NOTE: Make sure the array is "Optimal" first!

[codesyntax lang="bash"]

/opt/megaraid/bin/MegaCli64 -LDInfo -Lall -aALL | grep State

[/codesyntax]

State : Optimal

List physical drives info:

[codesyntax lang="bash"]

/opt/megaraid/bin/MegaCli64 -PDList -a0
/opt/megaraid/bin/MegaCli64 -PDList -aALL | grep -e '^$' -e Slot -e Count -e Enclosure

[/codesyntax]

Here is the magic command to silence the alarm:

[codesyntax lang="bash"]

/opt/megaraid/bin/MegaCli -AdpSetProp -AlarmSilence -aALL

[/codesyntax]

Blink the LED on drive:

[codesyntax lang="bash"]

# to start
/opt/megaraid/bin/MegaCli64 -PdLocate -start -physdrv[E:S] -aALL
# to stop
/opt/megaraid/bin/MegaCli64 -PdLocate -stop -physdrv[E:S] -aALL

[/codesyntax]

Take the disk offline:[codesyntax lang="bash"]

/opt/megaraid/bin/MegaCli64 -PDOffline -PhysDrv '[E:S]' -a0

[/codesyntax]

Mark the disk as missing:

[codesyntax lang="bash"]

/opt/megaraid/bin/MegaCli64 -PDMarkMissing -PhysDrv '[E:S]' -a0

[/codesyntax]

Prepare the disk for removal:

[codesyntax lang="bash"]

/opt/megaraid/bin/MegaCli64 -PDPrpRmv -PhysDrv '[E:S]' -a0

[/codesyntax]

Now you should replace the defective this!

Check the status:

[codesyntax lang="bash"]

/opt/megaraid/bin/MegaCli64 -PDList -aALL | grep "Firmware state"

[/codesyntax]

Firmware state: Rebuild
Firmware state: Online, Spun Up
Firmware state: Online, Spun Up
Firmware state: Online, Spun Up

Checking Drives Using SMARTCTL:

[codesyntax lang="bash"]

/usr/sbin/smartctl -a -d megaraid,2 -H /dev/sda

[/codesyntax]

/usr/sbin/smartctl -a -d megaraid,2 -H /dev/sda
smartctl 5.41 2011-06-09 r3365 [x86_64-linux-3.2.0-4-amd64] (local build)
Copyright (C) 2002-11 by Bruce Allen, http://smartmontools.sourceforge.net

Vendor:               SEAGATE
Product:              ST3600057SS
Revision:             ES66
User Capacity:        600,127,266,816 bytes [600 GB]
Logical block size:   512 bytes
Logical Unit id:      0x5000c5005ef315eb
Serial number:        6SL5QTLM
Device type:          disk
Transport protocol:   SAS
Local Time is:        Wed May 20 05:56:05 2015 UTC
Device supports SMART and is Enabled
Temperature Warning Disabled or Not Supported
SMART Health Status: OK

Current Drive Temperature:     44 C
Drive Trip Temperature:        68 C
Elements in grown defect list: 0
Vendor (Seagate) cache information
  Blocks sent to initiator = 553885253
  Blocks received from initiator = 2109947912
  Blocks read from cache and sent to initiator = 774081696
  Number of read and write commands whose size <= segment size = 727594903
  Number of read and write commands whose size > segment size = 144
Vendor (Seagate/Hitachi) factory information
  number of hours powered up = 19261.48
  number of minutes until next internal SMART test = 11

Error counter log:
           Errors Corrected by           Total   Correction     Gigabytes    Total
               ECC          rereads/    errors   algorithm      processed    uncorrected
           fast | delayed   rewrites  corrected  invocations   [10^9 bytes]  errors
read:   235631546        0         0  235631546   235631546       8429.942           0
write:         0        0         0         0          0      27589.036           0
verify: 1419827501        9         0  1419827510   1419827511      64352.695           1

Non-medium error count:        0

SMART Self-test log
Num  Test              Status                 segment  LifeTime  LBA_first_err [SK ASC ASQ]
     Description                              number   (hours)
# 1  Background short  Completed                  32       2                 - [-   -    -]
# 2  Background long   Completed                  32       2                 - [-   -    -]
# 3  Background short  Completed                  32       1                 - [-   -    -]

Long (extended) Self Test duration: 6400 seconds [106.7 minutes]

Check and mark badblocks on ext4 partitions

Publicat la 20 May 2015 Niciun comentariu

My storage is acting weird today and I'm trying to fix it with this command:

[codesyntax lang="bash"]

fsck.ext4 -vcDfty -C 0 /dev/vg0/lv0

[/codesyntax]

And the result was:

/dev/vg0/lv0: ***** FILE SYSTEM WAS MODIFIED *****

        6329 inodes used (0.01%, out of 107380736)
          44 non-contiguous files (0.7%)
           4 non-contiguous directories (0.1%)
             # of inodes with ind/dind/tind blocks: 0/0/0
             Extent depth histogram: 6123/119
    86511679 blocks used (20.14%, out of 429497344)
         178 bad blocks
          40 large files

        5522 regular files
         719 directories
           0 character device files
           0 block device files
           0 fifos
  4294967278 links
           0 symbolic links (0 fast symbolic links)
           0 sockets
------------
        5965 files
Memory used: 676k/416k (284k/393k), time: 21291.73/25.22/ 0.17
I/O read: 89MB, write: 19MB, rate: 0.01MB/s

178 bad blocks marked!

Check if an IP is in a subnet

Publicat la 16 May 2015 Niciun comentariu

At some point I counted my DROP rules in my firewall and the result was kinda frightening. A lot of subnets and even more IPs...
What was really annoying was that there were a lot of IP addresses which belonged to an already blocked subnet, so I needed a script to check this for me.

It has to be a script to do this already out there in the wild. Also a machine is faster than a human. Having this in mind, why should I reinvent the wheel? So after searching a little bit on web, I found this nice perl script.

[codesyntax lang="perl"]

#!/usr/bin/perl

use strict;

use Socket qw( inet_aton );

sub ip2long($);
sub in_subnet($$);

my $ip = $ARGV[0];
my $subnet = $ARGV[1];

if( in_subnet( $ip, $subnet ) )
{
	print "It's in the subnet\n";
}
else
{
	print "It's NOT in the subnet\n";
}

sub ip2long($)
{
	return( unpack( 'N', inet_aton(shift) ) );
}

sub in_subnet($$)
{
	my $ip = shift;
	my $subnet = shift;

	my $ip_long = ip2long( $ip );

	if( $subnet=~m|(^\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})/(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})$| )
	{
		my $subnet = ip2long( $1 );
		my $mask = ip2long( $2 );

		if( ($ip_long & $mask)==$subnet )
		{
			return( 1 );
		}
	}
	elsif( $subnet=~m|(^\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})/(\d{1,2})$| )
	{
		my $subnet = ip2long( $1 );
		my $bits = $2;
		my $mask = -1<<(32-$bits);

		$subnet&= $mask;

		if( ($ip_long & $mask)==$subnet )
		{
			return( 1 );
		}
	}
	elsif( $subnet=~m|(^\d{1,3}\.\d{1,3}\.\d{1,3}\.)(\d{1,3})-(\d{1,3})$| )
	{
		my $start_ip = ip2long( $1.$2 );
		my $end_ip = ip2long( $1.$3 );

		if( $start_ip<=$ip_long and $end_ip>=$ip_long )
		{
			return( 1 );
		}
	}
	elsif( $subnet=~m|^[\d\*]{1,3}\.[\d\*]{1,3}\.[\d\*]{1,3}\.[\d\*]{1,3}$| )
	{
		my $search_string = $subnet;

		$search_string=~s/\./\\\./g;
		$search_string=~s/\*/\.\*/g;

		if( $ip=~/^$search_string$/ )
		{
			return( 1 );
		}
	}

	return( 0 );
}

[/codesyntax]

Source: http://www.mikealeonetti.com/wiki/index.php?title=Check_if_an_IP_is_in_a_subnet_in_Perl

Owncloud: How to reset users' password

Publicat la 15 May 2015 Niciun comentariu

I won't bore you with the details, so let's just say that for some reason I don't have the owncloud admin password anymore.
I have spend lots of time on owncloud forum and found few more solution. In my point of view the below one is very simple and effective.

  • Get the passwordsalt

[codesyntax lang="bash"]

server owncloud # grep passwordsalt config/config.php 
  'passwordsalt' => 'ZnuaO2o4s3Qydg5xvR4gk7yZQn7v.L',

[/codesyntax]

  • Prepare the "hack"

[codesyntax lang="bash"]

server owncloud # cd /tmp/
server tmp # wget -c "http://cvsweb.openwall.com/cgi/cvsweb.cgi/~checkout~/projects/phpass/PasswordHash.php"
server tmp # wget -c "http://cvsweb.openwall.com/cgi/cvsweb.cgi/~checkout~/projects/phpass/test.php"
server tmp # sed -e "s/$t_hasher = new PasswordHash(8, FALSE);/$t_hasher = new PasswordHash(8, CRYPT_BLOWFISH!=1);/g" -i test.php
server tmp # sed -e "s/$correct = 'test12345'/$correct = 'admin123'.'ZnuaO2o4s3Qydg5xvR4gk7yZQn7v.L';/g" -i test.php

[/codesyntax]

  • Run the test.php file

[codesyntax lang="bash"]

server tmp # php -f test.php 
Hash: $2a$08$sIE2IL4xZwADAqpdGeLY7.QOYBC01x7U3IKE/YS6XZ1n.TVd1jnTS
Check correct: '1' (should be '1')
Check wrong: '' (should be '0' or '')
Hash: $P$BdVJYUfc8uplEowbiO3WWPRKXLLLY..
Check correct: '1' (should be '1')
Check wrong: '' (should be '0' or '')
Hash: $P$9IQRaTwmfeRo7ud9Fh4E2PdI0S3r.L0
Check correct: '' (should be '1')
Check wrong: '' (should be '0' or '')
Some tests have FAILED

[/codesyntax]

  • Update the new password to admin user via MySQL query

[codesyntax lang="bash"]

root@localhost [(none)]> use owncloud;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
root@localhost [owncloud]> update oc_users set password="$2a$08$sIE2IL4xZwADAqpdGeLY7.QOYBC01x7U3IKE/YS6XZ1n.TVd1jnTS" where uid="admin";
Query OK, 0 rows affected (0.13 sec)
Rows matched: 1  Changed: 0  Warnings: 0
root@localhost [owncloud]>

[/codesyntax]

  • Now, you have successfully reset your owncloud password, just navigate on your owncloud installation url on browser and login into your admin account using your new password (in my case new password is admin123)

ldappasswd and "ldap_sasl_interactive_bind_s: Invalid credentials (49)" error message

Publicat la 15 May 2015 Niciun comentariu

Some context might be useful. We have an openldap instance to manage users. We also have phpLDAPadmin, but that's not the point. The point is that I want to add/edit an user from command line. Adding a user it not a problem.

[codesyntax lang="bash"]

ldapadduser john.doe users
Warning : using command-line passwords, ldapscripts may not be safe
Successfully added user john.doe to LDAP
Successfully set password for user john.doe

[/codesyntax]

However, changing the password was a little bit more problematic.

[codesyntax lang="bash"]

ldappasswd briana.bennett
SASL/DIGEST-MD5 authentication started
Please enter your password: 
ldap_sasl_interactive_bind_s: Invalid credentials (49)
	additional info: SASL(-13): user not found: no secret in database

[/codesyntax]

I also tried with:

[codesyntax lang="bash"]

ldappasswd -D "cn=admin,dc=domain,dc=net" -W -x john.doe
Enter LDAP Password:
Result: Invalid syntax (21)
Additional info: Invalid DN

[/codesyntax]

Hmm... have no fear, I solved the problem. For future reference if anyone happens across this post with the same issue, the user you are trying to change must also be a full DN:

[codesyntax lang="bash"]

ldappasswd -D 'cn=admin,dc=domain,dc=net' -W -S -x 'uid=john.doe,ou=users,dc=domain,dc=net' -s KZ1URpsdEhP1HOJG

[/codesyntax]

Note: instead of using -s (which is used to specify the password on the command line)  -S to instruct ldappasswd to prompt for new password.

FortiGate-200D VPN users and groups operations

Publicat la 5 May 2015 Niciun comentariu

Recently we bought a FortiGate-200D VPN box. I have more good things than bad things to say about this device.
Long story short. I had to remove some users and because of some voodoo type of problem I couldn't do it from UI (I will contact their support that's for sure), so I had to do it from CLI. Who worked with Citrix Netscalers will find FortiGate's CLI a piece of sh!t (documentation makes no exception), but that's a different story.

  • To display one or all users

[codesyntax lang="bash"]

fgw # config user local
fgw (local) # get | grep john.doe
fgw (local) # get john.doe
fgw (local) # get

[/codesyntax]

  • To delete a user

[codesyntax lang="bash"]

fgw # config user local
fgw (local) # delete john.doe

[/codesyntax]

 

Note: When you're receiving an error like the one bellow the user is attached to one or more user groups.
The entry is used by other 1 entries
Command fail. Return code -23

In order to remove the user you have two options:

  1. CLI:

    2. [codesyntax lang="bash"]

    fgw # config user group
fgw (group) # show
config user group
    edit "ssl-vpn_office_users"
        set member "user1" "user2" "john.doe" "user4" "user5"
    next
end
fgw (group) # edit "group_name"
fgw (group_name) # set member "user1" "user2" "user3" "user4"
fgw (group_name) # next 
fgw (group) # end

    [/codesyntax]

  2. UI:
    You will have to login to the FortiGate webinterface, navigate to User & Device > User definition, edit john.doe and uncheck Add this user to groups

haproxy: rewrite request aka prefix url

Publicat la 27 March 2015 Niciun comentariu

Today I had a task which sounded like this "Change for balancer urls for this VIP - vip:8080 -> server* 16081/int"

Long story short:

frontend my_vip
     bind x.x.x.x:8080
     mode http
     maxconn 30000
     default_backend my_backend
backend my_backend
     mode http
     balance leastconn
     acl int_prefix path_beg /int
     reqrep ^([^\ :]*)\ /(.*) \1\ /int/\2 unless int_prefix
     server server1 y.y.y.y:16081 maxconn 32 check fall 3
     server server2 z.z.z.z:16081 maxconn 32 check fall 3

(openssl) verify that a private key matches a certificate

Publicat la 17 March 2015 Niciun comentariu

A while ago I had to renew the SSL certificate for a website I'm taking care of.

How do I verify that a private key matches a certificate?
[codesyntax lang="bash"]

openssl x509 -noout -modulus -in server.crt | openssl md5
openssl rsa -noout -modulus -in server.key | openssl md5

[/codesyntax]

How do I verify that a CSR matches a certificate match?
[codesyntax lang="bash"]

openssl req -noout -modulus -in server.csr | openssl md5

[/codesyntax]

Enable LDAP authentication in Apache

Publicat la 5 March 2015 Niciun comentariu

Assuming you have a LDAP server somewhere and you don't want to authenticate users via htpasswd file anymore... I mean, having all your users in one place is a good thing - it's debatable, but in general is a good thing, right?

Now, the technical part...

My LDAP structure is like this:
- groups: cn=group,ou=groups,dc=example,dc=com
- users: uid=firstname.lastname,ou=users,dc=example,dc=com

Next... apache2...

[codesyntax lang="bash"]

a2enmod authnz_ldap

[/codesyntax]

Add this inside your virtualhost.

<Location />
        Order allow,deny
        Allow from all
        Deny from all
        AuthName "Boo..."
        AuthType Basic
        AuthBasicProvider ldap
        AuthzLDAPAuthoritative on

        # Search user
        AuthLDAPURL ldap://IP-DOMAIN-CONTROLLER:389/ou=users,dc=example,dc=com?uid

        # Use this user to bind to LDAP
        AuthLDAPBindDN "uid=ldapauthuser,ou=users,dc=example,dc=com"
        AuthLDAPBindPassword "password"
        Require valid-user
        Satisfy any

        # More restrictions!
        # specific user
        #   Require ldap-user john.doe john1.doe john2.doe
        # specific user by DN
        #   Require ldap-dn CN=John Doe,OU=Finance,OU=Germany,DC=example,DC=net
        # member of group
        #   Require ldap-group CN=Finance Department,OU=Finance,OU=Germany,DC=example,DC=net
</Location>

Restart apache server

[codesyntax lang="bash"]

/etc/init.d/apache2 restart

[/codesyntax]

That's it!

virtualbox: Disable DHCP server for host-only network

Publicat la 24 January 2015 1 comentariu

Recently I decided to install virtualbox headless on my server to have something to play around with. Ok... cool...
The installation went fine, creation couple of VMs also.

Also I wanted to have my DHCP server, controlled by me. So far so good. I installed isc-dhcp-server on host, I configured it just fine... but at some point I realized that there is something wrong in the logs.

Jan 24 02:11:00 server dhcpd: DHCPDISCOVER from 08:00:27:3f:03:fe via vboxnet0
Jan 24 02:11:00 server dhcpd: DHCPOFFER on 192.168.56.253 to 08:00:27:3f:03:fe via vboxnet0
Jan 24 02:11:00 server dhcpd: DHCPREQUEST for 192.168.56.106 (192.168.56.100) from 08:00:27:3f:03:fe via vboxnet0: lease 192.168.56.106 unavailable.

Hmm! Houston we have a problem! After digging a little bit, I discovered that there is rogue DHCP server in my network... At least now I know what I am looking for.
I stopped my DHCP server, I disabled virtualbox DCHP server (from phpvirtualbox... I know, I was lazy) and my VM was still getting IP address... Something is fishy here. Ok, then maybe something is wrong with phpvirtualbox! I decided not to trust GUIs, so I started with to play around with cli commands:

[codesyntax lang="bash"]

vboxmanage list dhcpservers

[/codesyntax]
NetworkName:    HostInterfaceNetworking-vboxnet0
IP:             192.168.56.100
NetworkMask:    255.255.255.0
lowerIPAddress: 192.168.56.101
upperIPAddress: 192.168.56.254
Enabled:        No

Note: And this was the output after disabling the DHCP server from phpvirtualbox.

No problem...

[codesyntax lang="bash"]

vboxmanage dhcpserver modify --netname HostInterfaceNetworking-vboxnet0 --disable
vboxmanage list dhcpservers

[/codesyntax]
NetworkName:    HostInterfaceNetworking-vboxnet0
IP:             192.168.56.100
NetworkMask:    255.255.255.0
lowerIPAddress: 192.168.56.101
upperIPAddress: 192.168.56.254
Enabled:        No

Ok... I checked it one more time...
[codesyntax lang="bash"]

ifdown eth1 && ifup eth1

[/codesyntax]

The VM received the same IP address! This is going to be though, I said to myself...

[codesyntax lang="bash"]

VBoxManage dhcpserver remove --netname HostInterfaceNetworking-vboxnet0

[/codesyntax]

And now, when I listed the dhcpservers the above command returned nothing.

Perfect... I checked it again:

[codesyntax lang="bash"]

ifdown eth1 && ifup eth0

[/codesyntax]

Still got the IP... Fuck!!!

Ok... This is my personal server, is not in production, I can afford to reboot it... (Please notice that I was lazy again... I could remove and add back the virtualbox kernel modules)

Guess what?! After rebooting my VM was able to get the right IP address from my DHCP server!