Vim

I have a large mysqldump backup file of all databases on a server but I need a sql backup for a single database. So I wanted to break up the dump file into smaller pieces. This is one solution I came up with using vim.

1. I opened the dump file and a newly created file for the single database:

vim db-dump.sql newdumpfile.sql

2. I searched for the start of the database section by searching for:

/USE `databasename`

where “databasename” is the specific database I want to grab.

3. Once I was at the correct spot in the database dump I noticed that each individual database section starts with a comment:

---

Current Database: `databasename`

Once again, “databasename” is a specific database name.

4. So I yanked all lines until the next database section with the following vim command:

y/Current Database/

This tells vim to yank all lines until it reaches the next match for the regular expression “Current Database”. Vim will report how many lines have been yanked.

5. Next, switch to the new file and paste the lines that were yanked.

:bn

p

The vim command :bn, means switch to the next buffer. You could also
run the command “:e newdumpfile.sql” to create and edit a new sql file

The next vim command “p” is the paste command, it pastes into the
current buffer the last yanked hunk of text. In this case it would be
the lines you just yanked, the single database dump.

6. Finally save the changes to the new file with :wq

7. If you’d like to reload the database into mysql use the following:

mysql -p < newdumpfile.sql

Basic Drush

// Download a module or theme
drush dl cck

// Enable module or theme
drush en cck

// clear cache all, don't need sudo
drush cc all

// Get help message on command
drush help pm-update

// List site aliases
drush sa

// Get status on particular site, via site-alias
drush @math status

Advanced Drush

Using Drush to sync a remote server

Sync Dev Server with Staging Server

// drush core-rsync <source&gt; <destination&gt;
drush core-rsync @dev @stage

Download drushextras

drush dl drush_extras

Drush pushkey

This command creates a ssl public/private key in your home directory and copies the public key to a remote server. Subsequent drush commands will use this public/private key for authentication.

// okay substitute your username for "boris" and your server for "badinoff.com"
drush pushkey boris@badinoff.com

Drush Git Backup

Install DGB

cd <drush directory&gt;/commands/
git clone https://github.com/scor/dgb.git

Setup DGB for a site

# from DRUPAL_ROOT 
# cd up one directory
cd ..
# create directory for backed up database
mkdir databases
# initialize a git repository
git init

Back up Site

cd <DRUPAL_ROOT&gt;
drush dgb-backup

What is in DGB Archive

The DGB instance directory is the directory up from your DRUPALROOT.

each site’s db dump is in databases/

Database dump for default is in databases/default.sql

There is a Git repository of your site’s files

# from DRUPAL_ROOT cd into the DGB instance directory
cd ..
# see list of git commits
git log
# see what files have changed since last call to drush dgb-backup
git status

Clone the Site for development

# from DRUPAL_ROOT
cd ../..
# create a dev directory parallel to other site
mkdir dev
cd dev
git clone <path to dgb instance&gt;
# create a new branch called "my-dev-branch"
git branch my-dev-branch
# checkout the new dev branch
git checkout my-dev-branch
# now create a new database with the database dump
# change <dbuser&gt;, <dbpasswd&gt;, <devdbname&gt; and <path-to-dgb-instance&gt;
# and also the substitute default.sql for the name of your databasedump
mysql -u <dbuser&gt; -p <dbpasswd&gt; <devdbname&gt; < <path-to-dgb-instance&gt;/databases/default.sql
# now finally update settings.php to point to your dev database
vim sites/default/settings.php
# finally check that there is an .htaccess file in your new dev
# <DRUPAL_ROOT&gt; if not, copy it over from the cloned siteF

Another Way to Copy @live to @dev

# Core-rsync copies all of the files from one site instance to another.
# The first time you sync you can use --include-conf option to copy settings.php.
drush core-rsync @live @dev --include-conf
# Next change database settings to use @dev database.
vi <document-root&gt;/sites/default/settings.php

# After first sync use w/out --include-conf and it will preserve the existing settings.php.
drush core-rsync @live @dev

# Sync databases.
drush sql-sync @live @dev

I was printing out the sizes of different databases and trying to sum them. The command I was using was:
$ du -s /var/lib/mysql/* | sort n
...
6040    u_labs2
7632    dnvtech3
7828    courtwatch1
19664   statusnet2
23044   drupaldbc1
84896   managingnews0
248056  ibdata1
This prints out a two column list of files sizes and file names. I wanted to turn this into a csv. What is the best way to do this? AWK of course.

AWK to the Rescue

This print just the file sizes:
$ du -s /var/lib/mysql/* | sort n | awk '{print $1}'
This prints the file name first, comma, file size:
$ du -s /var/lib/mysql/* | sort n | awk '{print $2 ", " $1}'

u_labs2, 6040
dnvtech3, 7632
courtwatch1, 7828
statusnet1, 19664
drupaldbc2, 23044
managingnews0, 84896
ibdata1, 248056
You can either cut and paste the output into a spread sheet or redirect it into a csv like so:
$ du -s /var/lib/mysql/* | sort n | awk '{print $2 ", " $1}' > db-sizes.csv

Restart Apache2

sudo apache2ctl graceful

Continuously Monitor Apache Error Log

sudo tail -f /var/log/apache2/error.log

Apache Configuration

Add a new VirtualHost to the Apache Config file

sudo vi /etc/apache2/sites-available/default

A simple VirtualHost entry

<VirtualHost *:80>
  ServerName majorursa.net
  ServerAlias www.majorursa.net
  DocumentRoot /var/www/majorursalia/
</VirtualHost&gt;

A VirtualHost for a Drupal Site

<VirtualHost *:80>  
  ServerName majorursa.net
  ServerAlias www.majorursa.net
  DocumentRoot /var/www/majorursalia/drupal-6.22
  <Directory /var/www/majorursalia/drupal-6.22/>
    AllowOverride All
    Options None
    Order allow,deny
    Allow from all
  </Directory>
</VirtualHost>

Phusion Passenger

Do NOT start phusion passenger on a whim, forget about it and leave it Running!

  • Memory Hog!

layout: post
#+begin_src dot :file digraph2.png :cmdline -Kdot -Tpng  
digraph D {
  size="8,6"
 node [ shape = polygon,
  sides = 4,
  distortion = "0.0",
  orientation = "0.0",
  skew = "0.0",
  color = "#aaaaaa",
  style = filled,
  fontname = "Helvetica-Outline" ];
  apocalypse [sides=9 skew=".32" color="purple"]
  apocalypse -> zombie
  apocalypse -> zombies
  shovel [skew=".56" color="#aa2222"]
  subgraph singular {
    label="one"
    color=purple
    zombie -> shovel [color="#440000"]
    shovel -> run
  }
  run [sides=9, color=salmon2];
  subgraph plural {
    label="many"
    color=red
    zombies -> run [color="#00a4d4"]
  }
}
#+end_src
Then inside the buffer you can evaluate the code with “C-c C-c”, and you can see the results of evaluating the code with “C-c C-o”. This is made possible by Org-babel, a cool tool that allows you to run scripts from different languages in a single Org-mode buffer. Not only that but you can pipe output from one code block to another code block written in a different language. I will have more blog posts about this in the future. Org-babel is a part of Org-mode since Org-mode 7.x or so. Exciting stuff!

Further Reading


Org-babel Documentation
Graphviz
Edited with the emacs-chrome plugin

I updated the Scuttle firefox plugin to work with Firefox 4. This was the one utility that was holding me back from upgrading to Firefox 4. I created a github repository. for the project. Feel free to fork the project. Its a good simple plugin if you’d like to learn how to write one.

The original scuttle firefox plugin hasn’t been upgraded since 2007. The version in This updated version is compatible with Firefox 4. You can view the source or fork the project at my github repository. or you can download the plugin directly: scuttle-0.4.2-firefox+fl.xpi

Scuttle is an open source Website Bookmarking System. I switched to it for my bookmarking needs after Yahoo! announced they were shutting down delicious.com. The cool thing about Scuttle is that its open source, (its written in PHP.) so you can download the source and run it on your own web server and if there’s a new feature you want for your social bookmarking you can write it, also since it runs on your own server, you can be sure that the service will never end (just make regular backups.) The bad news is that you need to have a web server to run it. If you are just interested in an alternative to delicious you can try pinboard.in.

Credit for the original code goes to Marcus Campbell, developer of scuttle and the scuttle plugin and also to Andreas Keller for originally updating the scuttle plugin for Firefox 3.

Cloud Malaise

So, I’ve noticed recently I’ve been feeling uneasy about the cloud. I
think it stems from several factors.

  • The dissolution of Delicious.com.
  • The realization that the majority of traffic on the web goes to
    google, facebook, and twitter. And then realizing my patterns are the
    same.
  • Getting locked out of my google account for a few hours and then
    realizing how dependent I was on it.
  • Twitter going from open-source loving entity to starting to close
    access to its data.
  • The enormity of Facebook and its walled data, reminiscent of AOL’s
    walled garden.

These things led me to look at how I was using not just the cloud itself,
but services provided by humongous corporations. Corporations that
lock up my data, use it for data mining or even lose my data.

What I use Google for

Yeah, aside from everything.

I use google for:

  • Email
  • Documents
  • Search
  • Blogging
  • Calendar
  • Photos (although I mostly use Flickr.)
  • Google Maps
  • Android phone (does this count if I don’t use google’s apps?)

Wow, that’s kind of staggering. I think there’s a thing called
google-creep that happens, especially when you have an android phone
and you slowly switch to more and more google services. That happened
to me with Google Calendar and Picasa because of their android apps.

What can you do

Do I need to do anything? Me, personally? Probably yes. Just
realizing the dependence is

Update May 27, 2012

I’m leaving this post up for History; however, the currently recommended way to update drupal core and drupal modules, for minor version updates, is to use drush (the drupal shell).
cd [drupal directory]
drush pm-update
Drush pm-update will: check for updates for your drupal instance, inform you of the current status of drupal-core and all of your modules and it will ask if you want drush to update your drupal installation. You can answer yes or no. It is safe to check the update status, drush will not make any updates until you answer “y”. You can also check for security updates only with:
drush pm-update --security-only

Back to previous post

I found a couple of drupal sites on a web server that were badly in need of security updates. They are no longer being maintained by a developer but they are still in use by a user community so I didn’t like the idea of taking them down just because they are a security vulnerability, as they still have a great deal of value for people who use them. So I decided to update them with the patches that are available at fuerstnet.

If you follow the link above to fuerstnet, the site provides patches from all of the minor versions of drupal up to the most recent security level. So, if you check your drupal site’s status at “admin/reports/status” (in drupal 6). And you see that you’re drupal version is currently drupal 6.14. (That’s 4 security releases out of date!) Then you can go to fuerstnet and download the drupal-6.14-to-6.18.patch security fixes only patch, or you can download the 6.14-to-to-6.20.patch (which includes bug fixes + security fixes.).

The first thing you need to do is make sure you have a current back-up of the database and the drupal file system.

1. Quick back-up of the file system:
cp -R DRUPAL_ROOT DRUPAL_ROOT-bk
2. Copy of database:
mysqldump -uDB-USER -p DB-NAME > ~/DRUPAL-SITE-NAME.db-backup-DATE.sql

or you can do the database back-up with drush:
drush sql-dump –result-file=~/DRUPAL-SITE-NAME.db_backup-DATE.sql
Of course, you need to replace all of the ALL_CAPS names up above with the names that correspond with your site. Now on to the update procedure:

The procedure, as outlined at fuerstnet, is:
1. copy the patch into you’re drupal root directory.
2. put your site into maintenance mode. “admin/settings/site-maintenance” (in drupal 6)
3. cd into the DRUPAL_ROOT directory.
4. patch -p1 –dry-run < DRUPALPATCH
(replace DRUPALPATCH with the name of your patch, ie drupal-6.14-to-6.20.patch) if you do not see any errors in the output of the patch command, then you can run the command again for real.
5. patch -p1 < DRUPALPATCH
Next you have to run update.php, which if you have drush you can run like so:
6. drush updatedb
or you can visit http://YOUR-DRUPAL-SITE/update.php
7. next: test your site, update any modules that need updating and then don’t forget to pull your site out of maintenance mode.

Another possibility is to run the upgrades in drush with the shell command:
> drush pm-update –security-only

This command will update your site with any available security updates. It will first show you a list of updates that are available and ask if you if you want to proceed. You must answer y or n. This command is great, because you can run it in the root directory of any of your drupal sites and it will show you which updates are available. Also if you have site-aliases set up, you can run it against all of your aliases one at a time and you don’t have to leave the comfort of your /home directory!

> drush @dev pm-update –security-only
(where @dev is an alias name for one of your sites.)

Of course you can run the update command without the –security-only and it will offer to update all of your modules, themes and core drupal files.

[However I ran into trouble with this approach as it updated the themes as well on one of the sites I was updating and the original site developer had customized the theme instead of creating a new them and customizing that! So I restore the theme from backup. Of course when drush updates modules and themes it backs up the old ones in a directory outside the drupal directory, so if there is a mishap, such as overwriting a customized theme, you can restore and clear cache to restore the old theme. Although there is that safety net, I still now prefer to update security-fixes only at first, and then run non-security modules fixes one at a time and run the theme fixes manually. Schwew, big aside.]

After running the drush pm-update, which can be shortened to drush up, I like to run a quick:
> drush updatedb
to make sure the database has been updated properly, and:
> drush cc all
to make sure the caches are cleared.

And once you have all of your sites aliased, it is simple to ssh into your web server and run the drush up –security-only command once a week to check that all of your sites are up to date with the security fixes.

I recently bought a new large monitor, which opened the possibility of using Emacs with multiple, side-by-side windows. Which is brilliant by the way! Since I plugged it in, I’ve been “C-x 3”-ing like a madman. My new favorite way to learn programming is having an info file open in the left window and in the right window having a scheme (or python) prompt open. That is what I am doing now working on the Structure and Interpretation of Computer Programs. Luckily there was an info file available. I’m trying to think of a simple way to translate my programming epubs or pdfs into info files.

SICP with Emacs

Anyways on to todays topic. Emacs window commands!

The first thing to keep in mind, is the difference between windows and frames in emacs. To emacs: a window is any portal to a buffer. if you type in “C-x 2” in emacs and your view splits in half. These two portals are known as windows. What we usually call a new window, is known as a frame to emacs. So when you type in Ctrl-n in your browser and a new window pops open, in Emacsia, this is known as a new frame.

Open another window:

C-x 2 open another vertical window.
C-x 3 open another horizontal window.
C-x 4 b open a buffer in the other window
C-x 4 f open a new file in the other window.
C-x 1 close all other windows, leave current window open.
C-x 0 close current window, switch to other window.
C-x o move to other window

Positioning windows:

C-x + balance windows. This balances the size of all open windows.
C-x { widen current window.
C-x } narrow current window.
C-u C-x { widen current window by 4 columns.
C-u C-x } narrow current window by 4 columns.

Scroll other window:

M-C-v scroll other window

If you’ve ever played keywiz on Emacs: “M-x keywiz”, you were probably eating humble-hacker-pie by the end of the game. (For the uninitiated, keywiz is a game in emacs that’s asks 10 random questions about key-bindings for some times obscure emacs commands.) So far my top score is 4 (out of 10). So what is a novice to do? I’ve decided to write commands I want to learn on sticky notes on my monitor until I memorize them. So, without further ado, here’s this weeks list of new commands: M-g g goto line. This command will ask for a line number and take you there in the current file.

C-x v=

vc-diff. This command will show a diff between current file and HEAD in whatever Version Control System you are using.

C-x vv

vc-next-action. This command will perform the next appropriate VCS command, such as add current file to staging, or commit.

C-M-\

indent-region. If there is a defined region, this command will indent it.

C-M-;

comment-region. If there is a defined region, this command will comment the region out. (or uncomment a commented out region.) A note: to define a region. Do a C-[spc] at the beginning of the region and then move to the end of the region however you’d like (arrow keys, C-n, C-f, C-v). If you can’t see the regions add the following to your .emacs file: (transient-mark-mode 1). Second note, in the above key combination, capital C, means hold the Ctrl key, capital M, means hold the Alt key. If there are dashes between letters: type them together (or with C or M hold the Ctrl or Alt key while typing the other key. So with M-g g: Hold Alt while typing g, let go and then type g again.) Now off to work on a new version of keywiz, perhaps keywiz-jr!