[dm-crypt] Poor performances with nfs and Kernel 3.x

[Arno Wagner]

O.k., no such obervation with 3.2.2 on client and server. I got

root /gate/tmp>cat ttt | wcs > /dev/null
 read:   2.147 GB [   2147483648 B]    avg:  69.274 MB/sec [    31 sec]

(wcs, a.k.a. wc-stream is a small tool I wrote to do real-time
monitoring of pipeline throughput and byte count, sources
below.) This is with some ramp-up and almost 100% CPU load
on the (slower) server. 

I noticed though that nfsiostat is not the right tool to measure, as
it gives you performance over the whole time the device has been 
mounted, i.e. the throughput number keeps being updated in
real-time. 

Maybe re-run with wcs as shown above (compile instructions are in
the header of wcs.c) and also monitor CPU usage on the server
while this is rrunning, e.g. with "top". I am not saying you 
imagine the issue, but lets be sure the measurement is good.

If you do not want to compile anything, you could also use
something like "time cat ttt > /dev/null" and calculate
throughput manually. 

Arno


----- wcs.c -----

/* "wc-follow": wc with incremental output every second or so. 
 * Line positioning is done like in dd_rescue with 
 * "optimistic positioning" i.e. the hope that we have a terminal that 
 * understands vt100 positioning.
 *
 * (C) Arno Wagner <arno at wagner.name> 2008. Distributed under
 * The GNU public license v2
 *
 * Version 1.0
 *
 * Compile with "gcc -O6 -o wcs wcs.c"
 */

#include <stdlib.h>
#include <stdio.h>
#include <errno.h>

const char* up = "\x1b[A"; //] 
const char* down = "\n";
const char* right = "\x1b[C"; //]

char * usage() {
  return("\n"
"  Prints out running count and rate statistics about the data\n"
"  read on stdin to stderr. Does use cursor positioning, which\n" 
"  should work on most terminals (vt100 or later).\n"
"\n"
"  stdin is copied through to stdout, much like in tee.\n"
" \n" 
"  This programm does not support any commandline arguments.\n"  
"\n"
"\n");
}

void printlong(long long l) {
      if (l < 1000000L) 
        fprintf(stderr, "%7.3f kB", l/1000.0);
      else if (l < 1000000000) 
        fprintf(stderr, "%7.3f MB", l/1000000.0);
      else if (l < 1000000000000LL) 
        fprintf(stderr, "%7.3f GB", l/1000000000.0);
      else 
        fprintf(stderr, "%7.3f TB", l/1000000000000.0);
}  

int main(int argc, char ** argv) {
  char buf[4096];
  long long cnt = 0;
  time_t to, tn, ts, dt;
  int read_count;
  double rate;

  if (argc > 1) {
    fprintf(stderr, "%s", usage());
    exit(1);
  }

  to = time(NULL);
  ts = to;
  fprintf(stderr, down);

  while (1) {
    read_count = read (0, buf, sizeof(buf));
    if (read_count < 0 && errno == EINTR)  continue; // not an error
    if (read_count < 0) {
      perror("Abnormal condition in read from stdin:");
      exit(1);
    }

    if (read_count > 0) {
      cnt += read_count;
      if (fwrite (buf, 1, read_count, stdout) != read_count) {
        perror("Error in wcs writing to stdout:");  
        exit(1);
      }
    }

    tn = time(NULL);
    if (tn != to || read_count == 0) {
      to = tn;
      fprintf(stderr, "%s read: ", up);
      printlong(cnt);
      fprintf(stderr, " [ %12Ld B]    avg: ", cnt);
        
      // Calculate the rate
      dt = tn - ts;
      rate = cnt / (double)dt;
      printlong((long long) rate);
      fprintf(stderr, "/sec [ %5d sec]%s", dt, down);
    }
    if (read_count == 0)  break;                     // we are done 
  }
  fprintf(stderr, "\n");
}




-- 
Arno Wagner, Dr. sc. techn., Dipl. Inform., CISSP -- Email: arno at wagner.name 
GnuPG:  ID: 1E25338F  FP: 0C30 5782 9D93 F785 E79C  0296 797F 6B50 1E25 338F
----
One of the painful things about our time is that those who feel certainty 
are stupid, and those with any imagination and understanding are filled 
with doubt and indecision. -- Bertrand Russell