Upgrades and good grades

Matthew Heick Leave a comment

What is now:

 15:39:17 up  5:27,  1 user,  load average: 0.01, 0.04, 0.01

Ah, the uptime. One of the things we aspire to make as large as possible, and love every minute of it.

I did do a reboot today due to a mass of updates that I’ve lacked to do for 250+ days. Regardless, it’s a fresh uptime, and i’ll go with that.

With all these mass updates included SSL attacks such as poodle and heartbleed. I don’t typically run my server on https for public facing stuff, but right now I do for specific URLs and all that is presented is a self-signed certificate. You can go ahead and try https://www.unliterate.net to get the typical browser warnings.

So, with all the updates ssllabs SSL Server Test has given me a “T” (or A-), which I’m pretty proud of after reconfiguring. Maybe I’ll end up buying that cheap SSL cert and going for broke.

What used to be:

I happened to get really curious and find out if some old websites and documents existed from when I was originally fumbling around computers myself. Lo and behold, yes, I found ’em.

RBIL / Ralf Browns Interrupt List (wikipedia, cmu)

The de-facto bread and butter of my machine language learning. For every piece of hardware that downloaded its ROM into RAM, or any software that made hooks into the IVT, this list was just awesome.

I can’t recall how I located it back in the day, but what I do remember is that I was excited to get the updates to it online. Back in the modem days i’d wait upwards to 5 minutes to download 1 of the zip files, and then maybe an entire minute to load one of the text files into Windows 95’s notepad.

This list also got me into direct port access programming. Some of the interrupts and combinations needed for RS232 programming seemed slow to me, especially when trying to go faster than 9600 baud, so I had to turn to a different reference to learn to actually drive the serial controller.

Beyond Logic (retired)

Craig Peacock wrote awesome manuals on how to talk to the RS232 controller (specifically the 8250 and 16450/16550 UARTS), and also the Parallel Ports as well. It wasn’t until I read his manual about the parallel ports that realized that the bidirectional capability had quite a faster transfer rate over the cable than serial. His manuals helped deepened my knowledge on “how things worked”, cause who wouldn’t wanna know how things worked.

PHG Opcode (phg.chat.ru)

From Ralf Browns INTERRUP.LSTs came OPCODE.LST, which was a separate list created and maintained by Alex Potemkin. This list itself, when read entirely, gives you so much in-depth knowledge on how a processor works. From Intel and AMD, to Cyrix, you got instruction times, bugs, incompatibilities, and more than the whole nine yards. It was from this that I understood that 0F A2 means “Identity Yourself!”

From my memory this actually used to be at www.chat.ru/~phg, but as times change URLs have to change.

In a nutshell:

It’s been 20+ years that I’ve been using a keyboard and digging into computer guts, both software and hardware. I’ve been in and out of technology-related occupations, stepped into many hats, and accomplished so much, and I feel good about it.

Sometimes it feels good to take a step back and wonder how you got there, cause all you see is the progress you’ve had and know there is more to accomplish.

email.heick.email

Matthew Heick Leave a comment

Yays!

Today i’ve done it. I’ve acquired my email dream!

I’m talking about a nearly-automated @heick.email

…to explain…

For years, if you ever wanted to send me an email I would say:

“Just put your name at the front of @unliterate.net, and i’ll get it.”

Ever since I setup my email @unliterate.net wrong as a catch-all SPAM domain, I’ve had to deal with the following rigmarole when getting new email setup:

  1. Get the name@unliterate.net figured out
  2. Setup a folder in Thunderbird, my email client
  3. Setup a mail filter rule in my email client to move the received message from my Inbox to that folder

Which things have been working for years, and it’s been a fairly straightforward approach.

Until I got married, and we decided to get the heick.email domain and have all our mail shared.

…how to share the email…

So, the @unliterate.net experiment utilized POP3 as the server for me to get all my catch-all. This was easy to configure, and was fairly straightforward.

We needed to be able to share email across computers and devices, though, and still have the flexibility to do the “folder structure”-thingy that I’ve grown comfortable with the @unliterate.net “mail filter” rules.

So, the only solution is to go from POP3 to IMAP.

IMAP gives us the flexibility of folders on the server, not on the client. It also stores all the mail there, and the protocol (after swallowing that it’s not like POP3) is actually quite orderly and simple.

So, i’ve employed dovecot as my IMAP server, which was also my POP3 server, and configured it simply to enable IMAP and know where I want my users mailboxes to be stored.

After setting up DNS for mail, i’ve got email coming into @heick.email, and I’m happy.

I just need to be able to now do the mail filtering…

How to automate folder-making for IMAP

This was a basic challenge, and I love challenges.

I had to use my programming language of choice (PHP) with the php-imap module loaded, and fancy up a script that runs on a */1 (1 minute) cron task.

The script is as follows:

<?php
/**
 * The whole purpose of this script is to perform the following:
 * 1) Open an IMAP connection to an INBOX
 * 2) Look through all the messages
 * 3) Grab all messages and look for the first "To: " header in each message
 * 4) If the person in the "To: " is in the allowed domain
 * - We grab the user
 * - We check to see if their is a mailbox for that user, and move the messge there
 * - We delete the message
 * 5) If the person is not in the allowed domain
 * - We move the message to a default folder
 */

function get_imap_folders($resource, $config)
{
 // Get a list of mailboxes
 $original_folders = imap_listmailbox($resource, "{" . $config['server'] . ":" . $config['port'] . "}", "*");
 // these come through as {server:port}mailbox, so we just clean them up a bit
 $new_folders = array();
 $to_remove = "{" . $config['server'] . ":" . $config['port'] . "}";
 $folders = str_replace($to_remove, "", $original_folders);
 return $folders;
}

$config = array(
 'server' => 'localhost',
 'port' => '143',
 'username' => 'redacted',
 'password' => 'redacted',
 'folder' => 'INBOX',
 'spam' => 'SPAM',
 'debug' => true,
);
$debug_message = "";

$res = imap_open("{" . $config['server'] . ":" . $config['port'] . "/service=imap/novalidate-cert" . "}" . $config['folder'], $config['username'], $config['password']);
if (!$res)
{
 if ($config['debug'])
 {
 $debug_message = "IMAP Stream Failure";
 }
 die($debug_message);
}

$folders = get_imap_folders($res, $config);

// Lets get all the mail messages in the $config['folder']
$mbox = imap_check($res);
$number_messages = $mbox->Nmsgs;
if ($number_messages == 0)
{
 if ($config['debug'])
 {
 $debug_message = "No Messages";
 }
 die($debug_message);
}
$range = "1:" . $number_messages;

// now, we'll get the messages
$messages = imap_fetch_overview($res, $range);
foreach ($messages as $msg)
{
 $msgno = $msg->msgno;
 $to = $msg->to;

echo "Message: " . $msg->subject . "\n";
 if (strpos($to, "@"))
 {
 $array_to = explode("@", $to);
 $to = $array_to[0];
 }

// do we need to create a folder to move this message into?
 $destination_mbox = "{" . $config['server'] . ":" . $config['port'] . "}" . $to;
 if (!in_array($to, $folders))
 {
 if (imap_createmailbox($res, $destination_mbox))
 {
 echo "> Created folder [$to]\n";
 }
 else
 {
 echo "> Failed to create folder [$to]\n";
 }

}
 $folders = get_imap_folders($res, $config);
 if (imap_mail_move($res, $msgno, $to))
 {
 echo "+ Moved successfully\n";
 }
 else
 {
 echo "- Failed to move message\n";
 }
}
imap_expunge($res);
imap_close($res);
?>

To explain, basically this access my IMAP server, gets all the folders, then gets all the mail. It goes though the “to:” portions of the email addresses and sees if I have a folder that matches what’s in the name part of the email address in the “to:” portion. If it doesn’t exist, it makes the folder. Then, as a final result, it moves the mail to that folder and aborts.

So, this script now runs every minute, checking for new mail, creating the folders necessary and moving the messages.

vivre heick.email!

Basic DVD Ripping 101

Matthew Heick Leave a comment

Well, it came time where I got my Wedding DVD, and I had to convert to to Digital Video. I had to dig deep deep into my history, since I used to order DVDs from Netflix back in 2006, Rip them, convert them, and then enjoy them later on.

So, fast forward to 2017, from Windows XP to Windows 10, and awaaaaaaaaaaaaaaay we go.

Software Needed

SmartRipper 2.41

Optional: VLC Media Player from http://www.videolan.org

flaskmpeg from http://www.flaskmpeg.net/

XVid codec from https://www.xvid.com

Fraunhofer IIS MPEG Layer-3 Codec (professional) from the Radium MP3 “copy”.

What to do

  1. Install XVid and the MP3 codec. Fairly straightforward. Without these, you won’t be able to convert your DVD rip to a final copy.
  2. Install SmartRipper and VLC. VLC is only used by SmartRipper to help unlock copy-protected DVDs. Since my Wedding DVD wasn’t copy-protected, I didn’t really need this step.
  3. Extract flaskmpeg. Works out of the box.

Insert the DVD, run SmartRipper. In SmartRipper, you’ll need to isolate the actual film you want, make sure all Chapters and Cells are selected, and Rip. Sit and wait until it’s done.

Load up flaskmpeg, select the .ifo that would contain your DVD. flaskmpeg is smart enough to read it and ask you what you want of it. Select Output and Configure Output Module to select the XVid video codec and the MP3 audio codec, and configure as necessary. Once you’re all configured, FlasK It!.

Without configuring a codec in flask, you’ll get an uncompressed DVD rip. In my initial case, a 1GB MP2 compressed A/V file was 30GB uncompressed on my hard drive, and a laughable matter. After selecting the right options to my taste, a 1GB MP2 turned into a 450MB XVid/MP3 video.

Fruits of the labor

After all that done, I present to you:

Dovecot IMAP (part 1)

Matthew Heick Leave a comment

So, now i’ve gotta learn some IMAP with dovecot, which seems to be the go-to software for coolness.

Why? Easy!

I’m the proud new owner of heick.email, which currently is not pointed anywhere as far as web services or anything else. What I currently do for email is take *@unliterate.net, drop any messages to a SPAM folder, and let my Thunderbird pull the messages and based on the * part I sort it into folders. This way, I can use email addresses like matthew.is.awesome@unliterate.net, and I would get the email in my Inbox. It’d be considered SPAM, but if I wanted to actually draw my attention to it I’d create a mail filter and sort that into a folder.

One of the cool things about IMAP is the folders. You can, at a whim, create a folder and have client-related mail rules “filter” email to a separate folder. Folders can also be “private” or “shared”, which is a good feature.

So, i’ve set myself a goal so far:

  • Choose an IMAP server software (dovecot)
  • Configure it up with 143 (imap) and 993 (imaps)
  • Get POP (110) access
  • Setup the main SPAM inbox user
  • Setup scripts to be able to allocate an inbox to a user that is not part of SPAM (for some possible gifting or resale)
  • Setup /my/ user
  • …
  • profit

I am going to have to learn how to talk to IMAP via Terminal as well so I can fine-tune my configurations and see what I’m doing.

To start, I’m using Oracle VM VirtualBox with 2 VM’s

  • Windows XP for Outlook Express. Not caring about updates or anything
  • Centos 6.8 minimal, for base installation and configuration

usual “test” settings such as everything on the same network, root password is password, and no firewalls and full updates where necessary.

I’ve also created a user called userpoop with password poopuser. No domain definition yet, as this is the preliminaries.

$ adduser userpoop
$ passwd userpoop

IMAP 101

I’ve had to learn some basic commands to talk to IMAP, such as:

A login userpoop poopuser
B select INBOX
C logout

^ Some of the basics on getting in and out of your folder.

Installing on Centos6.8

yum does what I need it to do:

yum install dovecot

this gives me dovecot-2.0.9-22 and portreserve-0.0.4-11.

$ rpm -ql portreserve
/etc/portreserve
/etc/rc.d/init.d/portreserve
/sbin/portrelease
/sbin/portreserve
/usr/share/doc/portreserve-0.0.4
/usr/share/doc/portreserve-0.0.4/COPYING
/usr/share/doc/portreserve-0.0.4/ChangeLog
/usr/share/doc/portreserve-0.0.4/NEWS
/usr/share/doc/portreserve-0.0.4/README
/usr/share/man/man1/portrelease.1.gz
/usr/share/man/man1/portreserve.1.gz
/var/run/portreserve

…curious.

Everything with dovecot that I need is in /etc/dovecot/*. Keeping the default configuration, and knowing that the configuration loads via ASCII order i’ll just give myself a configuration file to work with:

touch /etc/dovecot/conf.d/99-self.conf

# First requirement of this is to ONLY have imap, not pop3
# So, we need to adjust the server that we're listiening on

# This enables imap(143) and imaps(993)
protocols = imap

# Setup where we store mail at
mail_location = mbox:~/mail:INBOX=/var/mail/%u

# HACKERY TO GET IT WORKING
# related to authentication
disable_plaintext_auth = no
auth_mechanisms = plain
# Using a passwordfile
passdb {
 driver = passwd-file
 args = scheme=CRYPT username_format=%u /etc/dovecot/users
}
userdb {
 driver = passwd-file
 args = username_format=%u /etc/dovecot/users
}

and I need an /etc/dovecot/users to get basic PLAIN authentication started, so I can get more familiar with this:

/etc/dovecot/users
userpoop:{PLAIN}poopuser:500:500::/home/userpoop

and now…we test via telnet

Numerical Nuptuals

Matthew Heick Leave a comment

I’ve been known for knowing when things have some rhythm or pattern to them.

#1 was 10/23/04 to 7/8/9, all unnaturally sequential in the sense of m/d/y

#2 was 1/15/14 to 2/29/16, the leap year Divorce, a 1:1461 chance every 4 years. And yes, there is also the “15 minus 1 equals 14” equation, too, but I believe that’s a pick-and-choose.

now, #3, occurring on 3/11/17, is all riddled with primes:

  • 3, 11, 17 are all prime numbers
  • The combinations of 311, 113, and 1117 are prime numbers as well.
  • albeit 31117 is not prime (29 2 • 37, the m/d/y format), 11317 is prime in the ordered set of d/m/y

The “prime” wedding, to say the least.