Email Address Password
Remember Me

Or Create a (Free) Account.
2004JanFebMarAprMayJunJul Aug Sep Oct Nov Dec
2005 Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
2006 Jan Feb Mar Apr May Jun Jul Aug Sep Oct Oct Oct
2007JanFebMarAprMayJunJulAugSepOctNovDec2007
2008JanFebMarAprMayJunJulAugSepOctNovDec2008
2009JanFebMarAprMayJunJulAugSepOctNovDec2009
2010JanFebMarAprMayJunJulAugSepOctNovDec2010
2011JanFebMarAprMayJunJulAugSepOctNovDec2011
2012JanFebMarAprMayJunJulAugSepOctNovDec2012
2013JanFebMarAprMayJunJulAugSepOctNovDec2013
2014JanFebMarAprMayJunJulAugSepOctNovDec2014

Mon 25th Oct 2010 @ 21:34 2010: /proc/self

I have just noticed a little oddity in /proc/self under Linux. The /proc/ pseudo filesystem exists in most (all?) Unixes and in Linux. Linux has a lot more stuff in /proc than Unix does, but the main thing in there, is data about all the processes currently in the process table. /proc/1/ belongs to the init process (which always gets PID #1). You can get your own process via the special "directory" entry called /proc/self. This will always be seen as a symbolic link to the running process's PID:

$ ls -ld /proc/self
lrwxrwxrwx 1 root root 64 Oct 25 18:21 /proc/self -> 28501

So we can see that the PID of that "ls" process was 28501.

However, we can also get our own PID from the shell builtin variable $$:
$ ls -ld /proc/$$
dr-xr-xr-x 7 steve steve 0 Oct 25 19:26 /proc/27659

That's fine, but one thing that struck me as odd, was the "fd" subdirectory. This lists all the open filedescriptors owned by that process (in the form, under Linux, of symbolic links, so you can see the data itself, but you can also find the path to that file):
$ ls -l /proc/$$/fd
total 0
lr-x------ 1 steve steve 64 Oct 25 19:26 0 -> /dev/pts/0
l-wx------ 1 steve steve 64 Oct 25 19:26 1 -> /dev/pts/0
l-wx------ 1 steve steve 64 Oct 25 19:26 2 -> /dev/pts/0
lrwx------ 1 steve steve 64 Oct 25 22:22 255 -> /dev/pts/0

Here I see that I have very few files open - 0 is stdin, 1 is stdout, 2 is stderr. Because this is an interactive shell, these are all linked to my terminal (pts/0). 255 (aka -1) is also open; I'm not sure why that is, to be honest.

Then looking at /proc/self/fd, that should show the same thing, right?
$ ls -l /proc/self/fd
total 0
lrwx------ 1 steve steve 64 Oct 25 22:28 0 -> /dev/pts/0
lrwx------ 1 steve steve 64 Oct 25 22:28 1 -> /dev/pts/0
lrwx------ 1 steve steve 64 Oct 25 22:28 2 -> /dev/pts/0
lr-x------ 1 steve steve 64 Oct 25 22:28 3 -> /proc/28504/fd

What's 28504? My PID is 27659. Also, fd #255 has disappeared, though I was never sure what that was about anyway.

After a bit of head-scratching, I finally realised that "ls -l /proc/$$/fd" gets the bash shell's details, whereas for "ls -l /proc/self/fd", "self" does not get directed to the shell's PID, but to ls's PID. Therefore /proc/self/fd shows that "ls" has been called with the argument "/proc/28504/fd" - investigating /proc/PID/fd of "ls /tmp" would show a link to /tmp for file descriptor #3!

Post a Comment               

Mon 25th Oct 2010 @ 14:56 2010: Causes of IO Wait

When a process can not execute because it is waiting for I/O to complete (rather than because it is waiting for a CPU to become available, or for the kernel to do something), it is classified as I/O Wait. In Linux, kernel/sched.c defines:

void __sched io_schedule(void)
{
struct rq *rq = &__raw_get_cpu_var(runqueues);

delayacct_blkio_start();
atomic_inc(&rq->nr_iowait);
schedule();
atomic_dec(&rq->nr_iowait);
delayacct_blkio_end();
}

which increments the nr_iowait count in the kernel. This is called by various I/O functions in the kernel, mainly related to disk I/O. Indeed, defines it as "Show the percentage of time that the CPU or CPUs were idle during which the system had an outstanding disk I/O request."

This is not the full story, however.

mm/mempool.c says in mempool_alloc:
/**
* mempool_alloc - allocate an element from a specific memory pool
* @pool: pointer to the memory pool which was allocated via
* mempool_create().
* @gfp_mask: the usual allocation bitmask.
*
* this function only sleeps if the alloc_fn function sleeps or
* returns NULL. Note that due to preallocation, this function
* *never* fails when called from process contexts. (it might
* fail if called from an IRQ context.)
*/
....... [ snip ]

/* Now start performing page reclaim */
gfp_temp = gfp_mask;
init_wait(&wait);
prepare_to_wait(&pool->wait, &wait, TASK_UNINTERRUPTIBLE);
smp_mb();
if (!pool->curr_nr) {
/*
* FIXME: this should be io_schedule(). The timeout is there
* as a workaround for some DM problems in 2.6.18.
*/
io_schedule_timeout(5*HZ);
}

kernel/sched.c has io_schedule_timeout which increments nr_iowait:

long __sched io_schedule_timeout(long timeout)
{
struct rq *rq = &__raw_get_cpu_var(runqueues);
long ret;

delayacct_blkio_start();
atomic_inc(&rq->nr_iowait);
ret = schedule_timeout(timeout);
atomic_dec(&rq->nr_iowait);
delayacct_blkio_end();
return ret;
}

sync_page() in mm/filemap.c also calls io_schedule(). So memory allocations can also cause IO Wait conditions
So memory allocations

Post a Comment               

Sat 23rd Oct 2010 @ 00:19 2010: 1.5 million pageviews in the past year

I messed up the Google Analytics about a month ago, so the best view of the past year that I have, is August 2009 - August 2010, which shows 1.5 million pageviews, or 4,000 per day, 171 per hour, 2.8 per minute. No desperate webserver optimisation required, but it's a pretty respectable figure. Click the thumbnail for a more detailed view.

1.5 million pageviews

Post a Comment               

Tue 19th Oct 2010 @ 00:57 2010: Nothing to Hide

I think that I have already linked to http://www.schneier.com/blog/archives/2007/07/privacy_and_the.html -it is not that, but the PDF that it links to, which contains the good stuff. I just wanted to highlight http://www.schneier.com/blog/archives/2007/07/privacy_and_the.html#c188996 - "Commenters here, for example, might not think that the fact that one's grandfather was jewish (or sunni, or a socialist) is particularly discreditable, but in other times or circles such information could get you killed."

This is exactly why I am resisting my daughter's school's plans to fingerprint her. If it doesn't need to be documented, then do not document it. If you do not need this level of detail, then do not store this level of detail.

Post a Comment               

Mon 18th Oct 2010 @ 00:25 2010: Interesting quote

"One can search the brain with a microscope and not find the mind, and can search the stars with a telescope and not find God." -- J. Gustav White

Post a Comment               

Sun 17th Oct 2010 @ 23:47 2010: Air Traffic Control


Air Traffic Game - http://www.freeonlinegames.com/game/air-traffic-chief.html

Post a Comment               

Sun 17th Oct 2010 @ 21:55 2010: OOo's put the willies up Microsoft



From http://www.theregister.co.uk/2010/10/14/microsoft_fears_open_office_org/ - as El Reg say, if Microsoft are attacking it, they are scared of it. Now that Oracle, rather than Sun, are behind OpenOffice.org, I'm sure they have every reason to be scared.

El Reg have a followup article here by Matt Asay, COO of Canonical, describing desktop office suites as "a market that five years from now no one will care about." Five years is a long time, but I doubt that we will have stopped using them.

Post a Comment               

Sun 17th Oct 2010 @ 21:47 2010: Firefox crashes

Ironically, this was Firefox 3.6.10...

firefox crash

Post a Comment               

Fri 15th Oct 2010 @ 22:37 2010: Shibboleet

Today's XKCD was a classic. I read it quickly this morning, and assumed that it was a misspelling of shibboleth, but no - in this context, shibboleet is far more appropriate.



The mouseover text is "I recently had someone ask me to go get a computer and turn it on so I could restart it. He refused to move further in the script until I said I had done that."

Post a Comment               

Wed 6th Oct 2010 @ 01:29 2010: Melodyne

http://www.youtube.com/v/jFCjv4_jqAY



Pulling single notes out of chords, changing their pitch and timing, just like editing a MIDI track. Awesome.

From the very end of that video:

With Direct Note Access it's possible for the first time to edit individual notes in polyphonic audio material

Possible applications range from the correction of errors to the complete refashioning of harmonies and rhythm

The first product to feature Melodyne's revolutionary Direct Note Access will be Version 2 of Melodyne plugin scheduled for Autumn 2008

This is dated 2008 ... this is already 2 years old.

Post a Comment               

Fri 1st Oct 2010 @ 01:11 2010: RMS for FSF on GPL

http://www.fsf.org/blogs/rms/assigning-copyright - RMS speaking for the FSF, offering very specific guidance on copyright assignment. As the article says at the end: "The FSF has had these texts reviewed by a lawyer, but you should get your own legal advice before using them."

Post a Comment               

Steve's urandom blog
Share on Twitter Share on Facebook Share on LinkedIn Share on Identi.ca Share on StumbleUpon
My Shell Scripting Book:
    Shell Scripting, Expert Recipes for Linux, Bash and more
is available online and from all good booksellers:


DefectiveByDesign.org