Perl Scripting And Developing Blog

print “Array : “, procsize() - $base; First we create a large number of scalars, each holding a random 100-character string. Then we create a hash with three-character keys and the same number and lengths of values, and populate an array similarly. After each step, we measure how much our process size has grown by looking at what psprints in the SZ column. (It may be in a different column on your system; on non-Unix boxes you’ll need to use whatever support your OS gives for measuring process size.) The output we got was Scalars: 1749 Hash : 682 Array : 592 Packing multiple values in a scalar avoids the overhead of creating a new one. So instead of push @array, $value; #… foreach $element (@array) { … } you can do $scalar .= “$separator$value”; #… while (($element) = $scalar =~ s/(.*?)($separator|$)//o) { … } (Or, if speed is an issue, cook up something a little wordier using substrand index. If you were forming many small arrays, then you can just turn each one into a scalar that you spliton $separator one at a time.) This is a last-ditch optimization that costs dearly in readability; comment the heck out of it if you have to use it. Don’t form unnecessary lists. Some operations generate large lists when all you want to do is go through the elements one at a time. Structured data can usually be parsed a line at a time: while () { … } rather than reading the whole file: { local $/; $file = ; } Although in the case of data that’s not structured by lines, that may really be your best choice. The worst of both worlds would be to use the wrong loop statement:
Note: If you are looking for cheap and inexpensive provider to host and run your tomcat application check Actions tomcat hosting services

timing 4 iterations of bare, memo… bare: 533 wallclock secs ( 0.70 usr + 0.65 sys = 1.35 CPU) memo: 11 wallclock secs ( 0.28 usr + 0.01 sys = 0.29 CPU) (warning: too few iterations for a reliable count) Four iterations is just enough to smooth out any discrepancies caused by random fluctuations, but the memoized chunk still executes fast enough to cause Benchmarkto warn us. In this case, the point has been already made. Do everything possible in Perl rather than calling external programs. Calling out to an external program may be de rigueur in shell scripts, but it costs time (spawning another process) that you often don’t need to expend in a Perl script. There is no need to call awk, sed, cut, head, grep, or similar text filters unless you’re not under time pressure and it happens to read better. Be aware of where the dividing line lies, though. While you can use modules to send e-mail for you, opening a pipe to a mail program that takes care of queuing it is likely to be both faster and easier to read. 11.3.2 Improving Memory Usage Just as the main CPU performance improvements are found in loops, the main memory gains are found in large data structures, so focus on how big your arrays and hashes get. Don’t try to economize by using scalars to save the data instead; that takes up even more space because of the overhead attached to any variable in Perl. Prefer hashes to scalars and arrays to hashes; here’s why: #!/usr/bin/perl -wl sub procsize { (split /s+/, `ps -lp $$ | tail -1`)[9]; # May not be 9 on your system } sub randstring { join ‘’, map chr rand 255, 1..100 } my $base = procsize(); eval “$$_ = randstring” for ‘aaa’ .. ‘zzz’; print “Scalars: “, procsize() - $base; $base = procsize; $hash{$_} = randstring for ‘aaa’ .. ‘zzz’; print “Hash : “, procsize() - $base; $base = procsize; push @array, randstring for ‘aaa’ .. ‘zzz’;
Note: If you are looking for best hosting provider to host and run your tomcat application check Astra tomcat hosting services

10.42 CPU) Memoizefunction calls with common arguments. Using the CPAN module Memoize(http://search.cpan.org/search?dist=Memoize) by Mark- Jason Dominus to transparently remember the results of any function call gives you an easy way to trade memory for speed. You could do the same thing with slightly less execution overhead in your program by creating appropriate data structures and checking for the existence of an entry there before calling a function, but Memoizeallows you to swap this capability in and out at the cost of a single line, and it can even save the results to disk to make them persistent. We can illustrate this with an example that looks up the canonical Internet host name for various servers. It might be quite common to call gethostbynamefor the same input more than once in, for instance, an application processing log files, and it’s reasonable to assume that within a short period say, a few minutes the results won’t change. Here’s a benchmark: use Benchmark; use Memoize; sub get_host_info { (gethostbyname shift)[0] || ‘undefined’; } sub get_hosts { for (1..shift) { my $true_name = get_host_info “www.$_.com” } } for ‘aaa’ .. ‘abc’; get_hosts(1); # Prime the local cache timethese (4, { bare => q{ get_hosts(10) }, memo => q{ use Memoize; memoize q(get_host_info); get_hosts(10) } }); For benchmark comparison purposes only, we first do a lookup on all the hosts so that the local nameserver has a chance to put the information in its cache. That stops the second benchmark from having an unfair advantage. Then we look up the host information for several hosts 10 times each. The results are:
Note: If you are looking for best hosting provider to host and run your tomcat application check Astra tomcat hosting services

We could have executed the script with dprofppand then interpreted the results by entering the single command dprofpp-pdirwalk; unfortunately, it would still have generated the tmon.out file. We would like to see an option for Devel::DProf to pipe its output into dprofpp to eliminate the temporary file. Warning: The tmon.out file can get extraordinarily large very quickly (the one for the previous test was 1.3MB). Before you run a large test, make sure you have an idea of how big it is likely to get by running smaller tests if necessary. Use Devel::DProfto profile programs for bottlenecks. 11.3 Making Things Better Here we discuss a few ways of improving your program’s use of resources. You’ll find terse descriptions of many more in Programming Perl (Chapter 8 in the second edition, Chapter 24 in the third). 11.3.1 Improving Execution Speed It is common for improvements in execution speed to cost dearly in the readability department. Adding lookup caches, rearranging statement order, and inlining subroutine calls conspire to make a program less maintainable; therefore, as we have said before, you do these things only when you have to. Once you’ve identified the code that is taking the most time, the next step is to optimize that code. A few suggestions follow. possible. Inside a loop, take every opportunity to get out of it as early as I was just now writing some code to parse a log file containing lines like 20010208001507:POP3- Server:[198.137.241.43]:gwbush The first statement in the loop was: next unless my ($year, $month, $day, $hour, $minute, $second, $user) = /(dddd)(dd)(dd)(dd)(dd)(dd):POP3- Server:.*:(.*)/; This may have been a natural translation of the obvious solution, but it was a tad slow on a 3 million line log file! Since many input lines did not
Note: If you are looking for cheap and quality provider to host and run your java application check Astra java hosting services

match this pattern, I was able to speed up the elimination of those lines by inserting the following line of code at the beginning of the loop: next unless index ($_, ‘:POP3-Server:’) > 0; (but not before making the mistake of leaving the parentheses out can you see why that was wrong?). This works because the string OP3-Server: still uniquely identifies the lines we want. We can do more; in general, simple string operations work faster than regular expressions, so we can follow that line of code with these statements instead of the one that originally began the loop: my ($year, $month, $day, $hour, $minute, $second) = unpack “A4A2A2A2A2A2″, $line; my ($user) = $line =~ /:POP3-Server:.*:(.*)/; However, this benchmarks no faster than the previous code. We can use one more piece of information about the line to get an improvement: the last characters before the username are ]:, and they don’t occur anywhere else on the line. Benchmarking this code shows a speed increase of 50% over the alternative: use Benchmark; $line = ; timethese (200000, { ‘unpack’ => q{ my ($year, $month, $day, $hour, $minute, $second) = unpack “A4A2A2A2A2A2″, $line; my $user = substr $line, index ($line, ‘]:’)+2; }, ‘regex’ => q{ my ($year , $month , $day, $hour, $minute, $second, $user) = ($line =~ /(dddd)(dd)(dd)(dd)(dd)(dd):POP3-Server:.*:(.*)/ ); } }); __END__ 20010208001507:POP3-Server:[198.137.241.43]:gwbush Benchmark: timing 200000 iterations of regex, unpack… regex: 15 wallclock secs (15.63 usr + 0.02 sys = 15.65 CPU) unpack: 10 wallclock secs (10.42 usr + 0.00 sys =
Note: If you are looking for cheap and quality provider to host and run your java application check Astra java hosting services

sub sym { $symlinks++ } sub dir { $dirs++ } sub regular { $regulars++ } sub sock { $sockets++ } sub block { $blocks++ } sub char { $chars++ } sub other { $others++ } This code is purposely broken out into subroutines rather than the more obvious data-driven solution of using a hash keyed by statflags to illustrate how Devel::DProf works. (Note that we again eschewed our usual brace style in the file type counting subroutines in favor of something more compact and more readable in this case.) You may be wondering why there is no mention of Devel::DProf in the source code; the answer is that this is a special module that is invoked by the -d option as an alternate debugger module. (So instead of the default module, which presents a command-line interface for stepping through your program, Devel::DProf basically intercepts subroutine calls[4] and starts a stopwatch.) This is how we run it: [4] Unfortunately, it is not possible to profile on a finer granularity than subroutines. $ perl -d:DProf dirwalk Symlinks: 1804, Dirs: 4519, Regulars: 67319 Sockets: 7, Blocks: 3467, Chars: 1856, Others: 50 This creates a file tmon.out (you can change the name with the environment variable PERL_DPROF_OUT_FILE_NAME) in the current directory and puts raw tracing information in it. The tool that can cook that information into something palatable for you is dprofpp: $ dprofpp Total Elapsed Time = 25.03237 Seconds User+System Time = 24.83237 Seconds Exclusive Times %Time ExclSec CumulS #Calls sec/call Csec/c Name 81.1 20.14 20.755 79022 0.0003 0.0003 main::process 28.8 7.157 146.90 4520 0.0016 0.0325 main::doit 9.05 2.248 1.678 67319 0.0000 0.0000 main::regular 0.56 0.140 0.102 4519 0.0000 0.0000 main::dir 0.36 0.090 0.061 3467 0.0000 0.0000 main::block 0.28 0.070 0.055 1804 0.0000 0.0000 main::sym 0.16 0.040 0.024 1856 0.0000 0.0000 main::char 0.04 0.010 0.010 1 0.0100 0.0100 main::BEGIN 0.00 0.000 -0.000 1 0.0000 -strict::import 0.00 0.000 -0.000 50 0.0000 -main::other 0.00 0.000 -0.000 7 0.0000 -main::sock 0.00 0.000 -0.000 1 0.0000 -strict::bits
Note: If you are looking for good and high quality web space to host and run your java application check Vision java hosting services

So make the code you benchmark as close as possible to what you will actually use in production. Make sure the machine is not heavily loaded, because a really busy machine could spend so much time keeping its house in order that it would skew the CPU figure. Therefore, make benchmarks short enough that you can keep an eye on the system with top or the equivalent while they’re running, and so you can run them again to ensure that the results are consistent. 11.2.3 Ask de Prof Our discussion of Benchmark presumes that you know where the critical regions of your code are. After all, if you have a subroutine that consumes only 1% of the time used by your program, no amount of optimization therein will improve your overall performance by more than 1%. One way of finding out where your program spends the most time is with the Devel::DProfmodule. To see it in action, we’ll construct a very simple program to count different types of file on our system: use strict; my ($symlinks, $dirs, $regulars, $sockets, $blocks, $chars, $others) = (0) x 7; doit (’/'); print <<"EOF"; Symlinks: $symlinks, Dirs: $dirs, Regulars: $regulars Sockets: $sockets, Blocks: $blocks, Chars: $chars, Others: $others EOF sub doit { my $dir = shift; opendir DIR, $dir or return; my @files = grep !/^..?$/, readdir DIR; closedir DIR; $dir = '' if $dir eq '/'; process("$dir/$_") for @files; doit($_) for grep ! -l, grep -d, map "$dir/$_", @files; } sub process { local $_ = shift; -l && return sym($_); -d && return dir($_); -f && return regular($_); -S && return sock($_); -b && return block($_); -c && return char($_); return other($_); }
Note: If you are looking for good and high quality web space to host and run your java application check Vision java hosting services

The results from this are:[3] [3] We’re leaving out some redundant parts of the output that would make it harder to read here. Benchmark: timing 400 iterations of loop, loopset, set, undef… loop: 21 wallclock secs (21.07 usr + 0.01 sys = 21.08 CPU) loopset: 21 wallclock secs (21.03 usr + 0.00 sys = 21.03 CPU) set: 17 wallclock secs (16.44 usr + 0.01 sys = 16.45 CPU) undef: 8 wallclock secs ( 7.58 usr + 0.00 sys = 7.58 CPU) Note that the tests are run in alphabetical order by name, not necessarily in the order they were specified; that’s because they’re hash keys, and hashes have no innate ordering. If the code you’re testing does little or no I/O, the figures you should compare are the CPU times; the wall clock figures can be skewed if the machine is busy servicing other processes. If you’re testing different code snippets that do lots of I/O, and do that I/O differently, then you have more of a problem. You have no choice except to compare wall clock figures, but you must make sure that the system isn’t interrupted by some other large process while you are benchmarking. So keep an eye on it with top or the equivalent to see what else is running during your benchmark. We can see that there’s no difference in setting an element to undefor 1, but there is a significant improvement in setting all the elements at once to 1 with a hash slice and an even bigger improvement in using a hash slice to set them all to undef (presumably because we aren’t taking the time to form a large list on the right-hand side). This tidbit can come in handy when we’re using a hash just to test for existence; we can set the keys using a hash slice without having to set the values and later test for existence with exists. Well, wait a minute. Might there be a performance hit from using the existsfunction rather than testing the truth of the hash value? Why guess when we can benchmark it? use Benchmark; my %h; @h{map int rand 1000, 1..100} = (1) x 100; timethese (1000000, { exists => ‘$x++ if exists $h{int rand 1000}’, true => ‘$x++ if $h{int rand 1000}’, }); We first fill the hash with up to 100 integer keys randomly chosen from the range 0 to 999, and then we compare the operation of testing an element for existence with testing one for truth. (The $x++ is just to give Perl something to do so we don’t have to worry about whether a smart optimizer might decide to elide our code altogether.) The results are
Note: If you are looking for high quality webhost to host and run your jsp application check Vision jsp hosting services

Benchmark: timing 1000000 iterations of exists, true… exists: 7 wallclock secs ( 6.52 usr + 0.02 sys = 6.54 CPU) true: 7 wallclock secs ( 6.28 usr + 0.00 sys = 6.28 CPU) There is no (significant) difference. Use the Benchmarkmodule to compare the relative speeds of different code strategies. What else can we learn from this Perl of Wisdom? Well, you may be wondering why there was no complaint in the first Benchmark example about the hash %h not being declared. We can infer from section 3.6.1 (pointing out that usestrict is lexically scoped) that Benchmark evaluates each chunk of code in a lexical scope that is separate from the one enclosing it. So if the code you’re testing is long enough to be worth strictchecking, put use strict at the beginning of each chunk. This scoping effect also means that any lexical variables you declare in the main program won’t be visible inside the benchmark code chunks. We should point out that there is rarely any point to benchmarking code that isn’t going to be run many, many times by your program. So if it’s just code that’s going to be run once, don’t bother benchmarking it unless it either (a) takes a really, really long time or (b) needs to run very, very quickly (e.g., has to respond to a user in real time). Benchmarking needs careful thought and can be an art. Benchmark the wrong thing, and the conclusion you draw can easily be the opposite of the right one. Suppose you’re benchmarking code that uses a constant function inside a loop. If you put the definition in the code chunk, it’ll get executed once for each iteration, whereas in your program it will get executed only once. The difference between the two approaches is seen in this benchmark, which shows how the result is changed by putting the constant definition in the loop: use constant OUTSTR => ‘The cat sat on the mat’; timethese(10000000, { inconst => q(use constant INSTR => ‘The cat sat on the mat’; INSTR =~ /cat/), outconst => q(OUTSTR =~ /cat/), }); inconst: 31 wallclock secs (31.27 usr + 0.00 sys = 31.27 CPU) outconst: 29 wallclock secs (28.78 usr + 0.00 sys = 28.78 CPU) Notice how we used the q() operator instead of ‘…’ to make a visual block out of the code chunks.
Note: If you are looking for high quality webhost to host and run your jsp application check Vision jsp hosting services

information from inside a script, or if your topis restricted for root’s use only, use the psprogram to get information about the process numbered pid. On SYSVish systems, use ps-lp pid; on BSDish systems, use psupid. (You can usually use a repeated -wflag on SYSV or a w flag on BSD to see more of the command column if it is truncated.) In each case, there are two columns labeled SZ(or VSZ) and RSS. Roughly speaking, these are, respectively, the total virtual memory in use by the process and the amount of physical memory in use, both in kilobytes. Note that the amount of memory allocated by a Unix process never decreases before it terminates; whenever it frees memory it makes it available only for the same process to reuse again later. (It is not returned to the free pool for other processes to use.) A very large and lengthy program of mine that reports its girth hourly has just displayed USER PID %CPU %MEM SZ RSS TT S START TIME COMMAND peter 28679 51.9 48.5 35248 30072 ? S 08:42:02 279:33 [truncated] which shows that its total memory usage is 35 MB, 30 of which are in physical memory. (The output was doctored slightly since the columns output by ps had actually run together.) Because it has had problems with running out of memory, this provides periodic statistics that tell me what it really uses. If you want to monitor the progress of your own Perl script, use $$ in place of pid: printf “Current virtual memory: %skn”, unpack ‘@24A5′, (`ps u $$`)[1]; (The actual unpack template you have to use may be different for your ps. By using a subscript on the backticks in list context, we can ignore the header line output by ps.) You can use ps like this to see the actual amount of memory used by your program as it executes. (And remember that on Unix, process sizes never decrease, no matter how much memory you free in your program. Your process will never get smaller than its largest memory allocation.) Under MacOS, access “About This Computer” from the Apple menu. A graphic displays the allocated memory per process and the fraction used (see Figure 11-1). Figure 11-1. Memory usage under MacOS
Note: If you are looking for reliable and quality webspace company to host and run your servlet application check Actions servlet hosting services