Before tweaking MySQL: it used 5% of the CPU and 250MB of the memory.

After tweaking, it uses 94.3% of the CPU and 2.33GB of the memory.

I could increase the memory use even more if I needed it.

I downloaded an ebook, it contains hundreds of one-page pdf files.  I certainly don’t want to upload all of these one-pagers to my Kindle. To concatenate these files, there is no simply and easy way.  Commercial software packages are available from $14.00 to 20Euro.

A simply perl hack does the job:

1. Install the perl module:
   perl -MCPAN -e 'install("PDF::Reuse")'
2. Create a perl program, call it “catpdf.pl”:
   
use strict;
use PDF::Reuse;

   prFile(”output.pdf”);

   for(@ARGV) {
   prDoc($_);
   }

   prEnd();

3. To concatenate, call it in two possible ways:

• perl catpdf.pl a.pdf b.pdf, or
• perl catpdf.pl *.pdf

————–

As a side note, it is really easy to concatenate mp3 files on mac, just do:
cat *.mp3>output.mp3

Just wrote the most complicated SQL query I’ve ever encountered.  Since the table “clicks” has more than 5 million records, I’m going to wait till tomorrow to get the result.

The reason I have to write this is that I was trying to use a PERL program to do the same, and so far, after two days, it is still running with very slow progress.


create table sqlhistory as
SELECT a.inc, a.clicktime, (SELECT MAX(clicktime) FROM clicks b
where a.keyid=b.keyid and a.adid=b.adid and b.clicktime<a.clicktime
) as bctime, a.keyid, a.adid, a.bid, a.price, a.rank, a.reserve_price, a.reserve_price_new
FROM clicks a
where a.bid<>(select c.bid from clicks c
where a.keyid=c.keyid and a.adid=c.adid and c.clicktime=(SELECT MAX(clicktime) FROM clicks b
where a.keyid=b.keyid and a.adid=b.adid and b.clicktime<a.clicktime
)
limit 1
)

======Update====

Sun Wei suggested a new way to do it, and the productivity increased almost 1000 fold

1.

create table clickorder as
select * from clicks
order by keyid, adid, clicktime

2.

create an auto_increment field called ‘inc’, and a new field called lastbid

3.

update clickorder a, clickorder b
set a.lastbid=b.bid
where b.inc=a.inc-1 and a.keyid=b.keyid and a.adid=b.adid


Browse youtube normally.  When you find a good video, simply add “kick” in front of the url: youtube.com.

e.g. http://youtube.com/vid=SuS76sksa

can be turned to:

http://kickyoutube.com/vid=SuS76sksa

Need to add the following lines to ssh config (~/.ssh/config or /etc/ssh/ssh_config depending on your desired scope)

ServerAliveInterval 300
ServerAliveCountMax 3

UPDATE: A web-based version is available. You don’t need to upload the whole file to do the conversion now.

Check out the new version at: http://blog.mikezhang.com/dcolumn/processtable.cgi.

Fortunately, there is a package called dcolumn that helps with this problem.

This dcolumn package “defines a system for defining columns of entries in an array or tabular which are to be aligned on a ‘decimal point’.”

Unfortunately, it is very hard to make the modifications manually on the tables to align the decimal points of numbers.  I searched online and could not find a solution.  So I created a web service that does this:

Basically, just visit the webpage: http://blog.mikezhang.com/dcolumn/.

There is only one field for you to browse a latex file.  Make sure it has extension .tex, otherwise, the system would not accept the file.

Click Submit, you will obtain the converted file with correctly aligned numbers in your tables.

See the screen shot of the finished page:

Now clicking “FILE” will bring you the new file, in txt format.  You need to change the extension to .tex to try it out.  This intended trouble helps you to avoid overwriting your original file.

Here is how it worked on a paper:

Before:

and After:

It’s that simple:

use DBI;
my $conn = DBI->connect
("DBI:mysql:db","owner","ownerpass") or die("Cannot connect: $DBI::errstr");
$sql=qq(select field from table where id=$id);
@array=@{$conn->selectcol_arrayref($sql)};


There are more than two ways to do it. I just put down two quick ways.

1. using reference to array
2. not using reference to array

Method 1:

In program:
processarray(\@array);
In sub processarray:
my $arrayref=shift;
my @array=@$arrayref;


Method 2:

In program:
processarray(@array);
In sub processarray:
my @array=@_;


Google blog posted an article on “Strengthening the Study of Computer Science”.

It started with:

At a time when more and more digital technologies are becoming indispensable to millions of people, the field of computer science (CS) is in trouble. Enrollment and retention of CS students, particularly those historically underrepresented in the field (women, African-Americans, Native-Americans, and Hispanics) has declined sharply.

I think the sharp decline in enrollment is really an issue, but the inner-economist of mine keeps saying that there is something wrong with the argument that we are able to address the issue by promoting this major to underrepresented people.

I believe in the invisible hand theory of resource allocation. There was definitely an excess in supply of CS students during the bubble years, but then the students shifted attention to other desciplines. We only have certain amount of brain power in the society, and the flow of these brains to different industries is definitely a good thing for the society as a whole. If computer science proves to be playing an ever increasing role in the society, I’m sure the brains will flow back. Just reward these brains accordingly.

Overall, the marginal decision maker should be indifferent to choose between any major if the market is effecient and can reward the students through the market system.