Ruminated Rumblings

Call for Delphi Developers

2009-11-10T17:08:00.000-08:00

My company is looking for a few good developers for some Delphi contract work. Are you (or someone you know) looking for a bit of work and fit one of the profiles below? If so, drop me a line at marshall dot fryman at gmail dot com. I'm looking for experienced Delphi developers only. You'll need references and samples. We only speak English so you will need to be reasonably conversant with it.

Digital Metaphors ReportBuilder specialist - We need over a hundred reports written. The reports are financially-oriented. We'll be doing Balance Sheet, Sales Summaries, Reorder Reports, etc. The focus here is mostly on the report writing and less on configuring the underlying RB engine. You'll need to be familiar with GL systems and accounting reports.

RemObjects specialist - I need someone who is very familiar with RemObjects SDK. We have the licenses but have run out of resources in the short-term so we're looking to contract a new 3-tier system based on the SDK. In our case, we will have a client, local server, and remote server (remote read as "over the Internet"). We will transfer data, configuration information and command-and-control requests over this conduit. We will also utilize Hydra on the local server to provide for plug-in extensions.

Accounting Integration - I am looking for an accounting developer who has experience in working with both the Sage SDK and the QB SDK to transfer accounting data from an external application to either Sage or QB products. We already have a full GL, we just need to push it to the appropriate accounting target.

Thanks in advance for any and all interest.

Clarification:
I have offices in Clearwater, FL and Anaheim, CA. There is no pressing need for you to be located near either of these sites. Remote workers are fine. My concern is the quality of the work not the location of the worker. I should also be clear I'm looking for contract workers at present. I may hire developers permanently mid-next year and would certainly discuss any opportunities with people I've contracted with before doing a general search.

D2007 and large text files

2009-07-31T10:31:00.000-07:00

Has anyone noticed that Delphi 2007 cannot append to a text file over 2GB in size? I freely admit that I had a problem in my code that created log files that large in the first place... but I also don't expect my compiler to blow up when dealing with files of any size.

My first warning was when the append command started throwing I/O Error 131 messages. Error 131 is negative seek. The only way that could be is if CG has a signed int32 it's using for the filesize. When I hit the upper bounds of int32, it just wraps around to a negative number.

In my case, I just fixed the bug and stopped making huge text files. Otherwise, it looks like I'd have to change how the append function works in the System library. Does anyone know if D2009 also has this problem?

Gmail, POP3 and Indy

2009-07-30T18:31:00.000-07:00

In my ever expanding need for dealing with the outside world, I've discovered a quirky little thing about Indy 10. When using the TIdPOP3 component with Gmail, you have to manually call DisconnectNotifyPeer or delete commands are ignored. I suppose this isn't quirky as much as not really documented any place I was able to find. Let's backup and start at the beginning, shall we?

I have a Windows service that polls a POP3 account looking for messages that it can import into a DB. It has all sorts of fun rules that can be assigned to discover what needs to be imported, what database it goes into, etc. At any rate, this code has been merrily chugging away on both Yahoo and SmarterMail servers with nary a hiccup. I recently needed to watch a Google domain app e-mail address and didn't think much of it.

Of course, in order to do so you have to implement SSL for POP3. As I've posted in the past, this is not really much of an issue, you simply create the TIdSSLIOHandlerSocketOpenSSL object and assigned it to the IOHandler like so:

Pop3Srv.IOHandler := TIdSSLIOHandlerSocketOpenSSL.Create(Pop3Srv);
Pop3Srv.UseTLS := utUseImplicitTLS;

Anyway, all was proceeding swimmingly until I retrieved the first message. In my case, I automatically delete any e-mail that matches my rules so it will not be imported a second time. This works fine with Yahoo and SmarterMail. You just call DeleteMessage(msgNumber) and voila, away it goes. With Gmail, no such luck.

Gmail actually has a three different settings under POP3. One keeps messages in the inbox after reading via POP, one archives it and one deletes it. Thinking this option was set wrong, I tried all three options... repeatedly... still no luck. I tried searching pretty extensively on this one and finally found the clue with RFC 1939. Gmail honors the "you must notify me you are quitting or I will not delete anything" rule that 1939 dictates.

When the client issues the QUIT command from the TRANSACTION state, the POP3 session enters the UPDATE state. (Note that if the client issues the QUIT command from the AUTHORIZATION state, the POP3 session terminates but does NOT enter the UPDATE state.) If a session terminates for some reason other than a client-issued QUIT command, the POP3 session does NOT enter the UPDATE state and MUST not remove any messages from the maildrop.

After checking the code for the IdPOP3, it turns out calling Free does not issue the QUIT command. It makes sense for it not to do so. What if you wanted to avoid the UPDATE state? It was just surprising that some mail servers worked without this command and some required it.

At any rate, I hope this helps someone else who's working with POP3 commands.

RAID, files and cloud storage

2009-04-17T16:08:00.000-07:00

RAID is a method that takes independent drives and lets a system group them together for security (redundancy or parity), speed enhancements, storage space increases or all three. One of the long-time stalwarts of the RAID environment is RAID 5. In RAID 5, you need at least 3 identically sized disks. They are combined so that the storage space is N-1 (i.e., in a 3-drive system, total space is 2x drive size). The last disk is used for parity. With a parity drive, you can remove any one of the drives and still have access to your data. If you remove two or more of the drives though, you'd better have a good backup.

How does this work? Through the magic of XOR. The following statements are all true:


A   XOR B   = PAR
PAR XOR B   = A
A   XOR PAR = B

That's how parity lets you lose one disk and still recover your data. The same rules also apply in larger sets. Notice that you have to rotate the position of the parity data though. So a 5 drive system looks like:


A   XOR B   XOR C   XOR D   XOR E   = PAR
PAR XOR B   XOR C   XOR D   XOR E   = A
A   XOR PAR XOR C   XOR D   XOR E   = B
A   XOR B   XOR PAR XOR D   XOR E   = C
A   XOR B   XOR C   XOR PAR XOR E   = D
A   XOR B   XOR C   XOR D   XOR PAR = E

The other interesting thing to note is that the size stored on each drive is 1/(n-1) of the total file size. Thus a 100 byte file in a RAID 5 system only stores 20 bytes on each drive. This is where the space increase comes from.

Now, the question is, why do we care? Other than it's nice to know how something works, this technique could be applied to cloud storage. If you follow some of the events that have happened with online storage providers, you may have seen a number of them come and go. The problem is, if they disappear or lose your data, what happens then?

If you are using them as a convenient off-site data storage pool for large amounts of infrequently used data, you could implement a RAID 5-style data split. Keep in mind there's a bit of overhead in doing so, but the point of this exercise is to reduce your dependence on any one provider. Coincidentally, it will likely cost you the same or less than using a single cloud storage provider.

Take for instance Amazon's S3 service and Rackspace's Mosso Cloud Files. Both services charge per gigabyte per month. For arguments sake, let's assume we have a 10GB RAR file (BIGFILE.RAR) that we want to backup. If we split it into a RAID 5, 3 drive format, that would leave us with BIGFILE.RAR.1, BIGFILE.RAR.2 and BIGFILE.RAR.RAID. Each file will be 5GB (10GB / (3-1)) in size. I can upload one file to Amazon, one file to Rackspace and retain one file on my hard drive. I've now backed up the file in an online form that can be recovered even if I lose my hard drive or Amazon / Rackspace has an outage at the same moment I need to have access to my data. So long as I have access to any two parts of the file, I can create the original output file.

Obviously, Amazon and Rackspace are large enough it is unlikely they'd actually lose the data over a longer period of time. The same can't be said of some the smaller players in the market. Companies like Streamload managed to wipe out about half of their customers data during a reorganization before finally closing their doors. Anyone caught unaware lost all of their data.

I should also note that you could use a PAR or PAR2 system. The easiest to use implementation is probably QuickPAR. It uses a similar system to what I've shown here but the source is not very conducive to Delphi developers. From what I can tell, it was originally developed to push binary files around Usenet, but it would work equally well for cloud storage. If you're just looking for a good off-the-shelf tool, QuickPAR is probably the way to go. If you're interested in developing a parity solution that you can embed in your code, the source is included below.


procedure File2RaidFiles(fileName:string; raidLength:integer);
var FS:TFileStream;
    outputFS:array of TFileStream;
    byteArray:array of byte;
    ix:integer;
begin
//test to make sure we were called correctly
if not FileExists(fileName) then
   raise Exception.Create(Format('File %s doesn''t exist', [fileName]));
if raidLength<2 then
   raise Exception.Create(Format('Raid Length must be greater than 1. Given value was %d',[raidLength]));

FS:=TFileStream.Create(fileName,fmOpenRead);
try
   setLength(outputFS, raidLength+1); //+1 for the parity byte
   setLength(byteArray, raidLength+1);
   for ix := 0 to raidLength do       //create an output location for each stripe (named .stripe#) and the parity file (named .raid)
      begin
      if ix=raidLength then
         outputFS[ix]:=TFileStream.Create(fileName+'.raid',fmCreate)
      else
         outputFS[ix]:=TFileStream.Create(fileName+'.'+IntToStr(ix),fmCreate);
      end;
   try
      while FS.Position<FS.Size do                         //while we haven't hit the end of the file
         begin
         FS.Read(byteArray[0],raidLength);                 //read in the bytes to the byteArray
         for ix := 0 to raidLength-2 do                    //this calcs the parity byte, it's calculated by XORing the other bytes to it
            byteArray[raidLength]:=byteArray[ix] xor byteArray[ix+1];
         for ix := 0 to raidLength do                      //write out the bytes to the respective stripes (stripes are < raidlength) and the parity file (outputFS[raidlength])
            outputFS[ix].Write(byteArray[ix],1);
         end;
   finally
      for ix := 0 to raidLength do                         //clean up the output streams
         outputFS[ix].Free;
   end;
finally
   FS.Free;                                                //clean up the input stream
end;
end;

procedure RaidFiles2File(fileName, outputName:string; raidLength:integer);
var FS:TFileStream;
    inputFS:array of TFileStream;
    testFS:TFileStream;
    byteArray:array of byte;
    ix, damage:integer;
    countOfMissing:integer;
    checkByte:byte;
    checkFails:boolean;
begin
//test to make sure we were called correctly
if FileExists(outputName) then
   raise Exception.Create(Format('File %s already exist', [outputName]));
if raidLength<2 then
   raise Exception.Create(Format('Raid Length must be greater than 1. Given value was %d',[raidLength]));
//init the basics
checkFails:=false;
testFS:=nil;
countOfMissing:=0;
damage:=0;
setLength(inputFS, raidLength+1); //+1 is the parity byte
setLength(byteArray, raidLength+1);
for ix := 0 to raidLength do
   inputFS[ix]:=nil;

//setup the output file stream
FS:=TFileStream.Create(outputName,fmCreate);
try
   //create the input file streams. make sure we pickup the count of missing streams and a pntr to an input stream for use later on (any stream will do, just testing for eof
   for ix := 0 to raidLength do
      begin
      if ix=raidLength then //this is the parity stream
         begin
         if FileExists(fileName+'.raid') then
            inputFS[ix]:=TFileStream.Create(fileName+'.raid',fmOpenRead)
         end
      else                  //this is a stripe stream
         begin
         if FileExists(fileName+'.'+IntToStr(ix)) then
            begin
            inputFS[ix]:=TFileStream.Create(fileName+'.'+IntToStr(ix),fmOpenRead);
            testFS:=inputFS[ix];  //this is just to test for eof. all files are the same size
            end;
         end;
      if inputFS[ix]=nil then inc(countOfMissing); //if we didn't get an input stream, add to the missing count
      end;
   //you are only allowed to have 1 missing input file
   if countOfMissing>1 then
      raise Exception.Create('Unable to recover file! You must have at least N-1 parts of the file');
   assert(testFS<>nil, 'testFS=nil? This should never happen');

   while testFS.Position<testFS.Size do //while we are not at the end of the input file
      begin
      if countOfMissing=0 then          //if there are no missing streams, we can just remerge the data together
         begin
         for ix := 0 to raidLength do
            inputFS[ix].Read(byteArray[ix],1);

         checkByte:=0;
         //calc a checkByte to make sure it all still agrees
         for ix := 0 to raidLength-2 do //0 based, stop 1 short of the end
            if ix=0 then
               checkByte:=byteArray[ix] xor byteArray[ix+1] //seed check byte by XORing the 1st two bytes together
            else
               checkByte:=checkbyte xor byteArray[ix+1];      //XOR the bytes together
         if checkByte<>byteArray[raidLength] then  //the new checkByte doesn't match the old parity byte. That means we have a problem, the file doesn't match
            checkFails:=true;
         end
      else                              //if there are missing streams, we have to calculate the missing data
         begin                          //you have to reverse the XOR with the parity bit to determine the result
         for ix := 0 to raidLength-1 do //not counting the end of the array, that's where we'll store the damaged byte
            if inputFS[ix]<>nil then    //if this is a valid stream, read it's data
               inputFS[ix].Read(byteArray[ix],1)
            else
               begin                    //this isn't a valid stream, so remember this is damaged and read from the parity stream for this position
               damage:=ix;
               inputFS[raidLength].Read(byteArray[ix],1);
               end;
         //this is calcs the damaged byte into the end of the array (raidLength)
         for ix := 0 to raidLength-2 do //0 based, stop 1 short of the end
            byteArray[raidLength]:=byteArray[ix] xor byteArray[ix+1]; //XOR the bytes together
         byteArray[damage]:=byteArray[raidLength]; //replace the damaged byte with the restored byte
         end;
      FS.Write(byteArray[0],raidLength);           //write out the merged (and psbly restored) data
      end;
finally
   for ix := 0 to raidLength do                   //clean up the memory from the input streams
      if inputFS[ix]<>nil then inputFS[ix].Free;
   FS.Free;                                       //clean up the memory for the output stream
end;
if checkFails then
   raise Exception.Create(Format('This file (%s) does not match to its parity checks. Damage may be present', [outputName]));
end;

Qt becomes LGPL licensed

2009-01-14T09:47:00.000-08:00

Nokia has announced that the new Qt library (4.5) will be available under the LGPL license (March 2009). Hopefully, this will mean that Embarcadero will revive Linux support with it's upcoming 64-bit, compile-to-many-targets compiler rewrite.

Actually, it could mean that we would finally have a true CLX revision that would let us have native write-once, compile-to-many-targets, cross platform support for Windows, Mac, Linux, Embedded Linux, Windows CE, and the S60 (eventually). If Delphi's component architecture could be revamped to produce true Qt widgets, Embarcadero would suddenly enable all of the Delphi component vendors to gain access to a client-base that is much larger than what Delphi currently offers. Qt does not have much selection from third-party component vendors. A DevExpress or TMS Software offering would provide a nice upgrade to the basic Qt widgets.

Delphi Prism theoretically supports multi-platform through the use of Mono but you have to add the managed code overhead to everything. There was a recent article on using Mono to develop for the iPhone which talked about stripping the managed code down to a true compiled state (Apple doesn't permit managed code to run apparently.) It was done with C# but there is no particular reason (that I'm aware of) Delphi Prism couldn't work the same way. Has anyone tried to do multi-platform with Prism?

Word, WordPad and RTF

2008-11-03T14:08:00.000-08:00

In yet another stunning victory for Microsoft's cross-compatibility, their RTF system is incompatible between their own products. Let me begin by explaining that I have a Delphi app that uses a mail merge system to merge database text into a document. Naturally, you can do this a lot of different ways but, for my particular instance, I need to merge from a non-ODBC compliant database and then automatically E-mail the resulting document to the correct person. This is a management tool we use fairly heavily.

In the past, I've always just used raw HTML formatting because it's handy and relatively standard. The particular request I've been working on is to format the resulting merge so that it can be printed in a "one summary page per item" format. HTML has the ability to do this with a style sheet as such:


<STYLE TYPE="text/css">
     P.breakhere {page-break-before: always}
</STYLE>

and use it via:


<p class="breakhere">

Thanks to HTML Goodies for the information.

Now, the downside to this is that HTML print formatting appears to be considered optional by... well... everyone. I initially sent the HTML to my Gmail account. It appears fine but without the page breaks. I would expect this considering Gmail has it's own HTML enclosure. I would not expect the PRINT to ignore all of the HTML formatting, but it does. I suppose this makes some sense in light of the header information they put at the top of the output. The process of placing the header must remove the rest of the print formatting. Keep in mind this is the equivalent of an HTML E-mail, not an HTML attachment. When I converted over to an HTML attachment, Gmail dutifully opens the attachment and FF3 prints the document out just fine.

On to Outlook. One would think that Outlook should honor HTML formatting since, unlike Gmail, it doesn't actually use any HTML for display or printing. This is not so. HTML prints from Outlook without the page breaks from the normal reader pane. Thinking I could force the issue, I dbl-clicked the message in Outlook (which of course puts it in Word) and voila - no difference. This is turning into a major hassle. Additionally, because Outlook is so clever, it won't really let you send an HTML attachment. Instead, it displays it in-line with the same problems printing as with an HTML E-mail.

On to the solution...

I had to select some format other than HTML but it must be 7-bit text-based because of the way my Delphi merge app works. That pretty much leaves me with PDF or RTF as easy standards that everyone can open. I settled on RTF because it's easy to create an RTF file in OpenOffice, save it out and hack it apart with TSE Pro. My merge codes are just enclosed in << >> (ala MS Word style) so a name would be


<<name>>

. Anyway, this all works, although I don't really recommend reading RTF as a form of entertainment. Many trial and error attempts later I had a document that works fine in WordPad and OpenOffice. It just won't work in Word.

This is not to say that Word doesn't open the document. It does, it just ignores the page breaks. I tried the same RTF file in Word 07 and Word 03 with the same results. Even more annoying, if I open the file in WordPad and then save it back out as RTF, Word still won't have the page breaks. After reading the RTF documentation, I was left with the impression that \page should always generate a new page. It turns out that Word won't take just a \page like the documentation, WordPad and OpenOffice. It HAS to be "\par \page \par " and please don't forget the ending space. In Word, this works fine. In WordPad and OO, it generates a leading blank line. In my case, I can call this close enough sine the summary pages are so short anyway. If I was pressed for space though, it would be an issue that would probably force me to PDF.

In another quirky little thing, when I created my file in OO and saved it out as RTF, my file size was 38k. If I open the file in WordPad and save it out, it shrinks to 13k. If I open the original in Word 07 and save it out, it grows to 87k. How can the same RTF encoding size vary that much from the same company? Even more annoying, why aren't WordPad and Word interoperable using RTF?

MapQuest and Delphi

2008-08-15T15:32:00.000-07:00

A few weeks ago I started down the path of discovering how to GeoCode addresses for Delphi. This is related to a project we had where we wanted to show a map of a given point and then show facilities located near it on a map. I had originally started working with maps.live.com (which is a MS site). They have a decent little interface for JS that seemed to work for what I wanted AND I didn't have to store the annoying little GeoCoding addresses anywhere. This works great if all you want is a single address. For instance, the code below works for single addresses:


<html>
   <head>
      <title>Address Lookup </title>
      <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
      <script type="text/javascript" src="http://dev.virtualearth.net/mapcontrol/mapcontrol.ashx?v=6"></script>
      <script type="text/javascript">

      var map = null;
      var findPlaceResults = null;
      function GetMap()
      {
         map = new VEMap('myMap:newmap');
         map.LoadMap();
         map.Find(null,'100 Enterprise Way, Scotts Valley, CA 95066', null, null, null, null, true, true, null, true, GetCoordinates);
      }

     function GetCoordinates(layer, resultsArray, places, hasMore, veErrorMessage)
     {
        findPlaceResults = places[0].LatLong;
        var myShape = new VEShape(VEShapeType.Pushpin, findPlaceResults);
        myShape.SetTitle('CodeGear');
        myShape.SetDescription('100 Enterprise Way<br>Scotts Valley, CA 95066<br>'+findPlaceResults.toString());
        map.AddShape(myShape);
     }

     </script>
   </head>
   <body onload="GetMap();">
      <div id='myMap:newmap' style="position:relative; width:800px; height:600px;"></div>
   </body>
</html>

The problem showed up when I went to add the facilities to the map. Thinking it would all work the same way, I added the same Map.Find code for each facility in the state of interest and voila. Except the voila was, "How come I only get one pushpin?" not "See how great this works!" It turns out that part of the answer is the MS VEMap object works asynchronously. I could live with that if the map moved around a little as it added pushpins... well, at the very least, I could just hide the map until it was done.

Unfortunately, MS also made the calls non-reentrant. This is actually what they document the problem as. To me, the code IS non-reentrant in that it never calls itself. Apparently, our definitions of re-entrant differ significantly. To MS, non-reentrant means that you have to wait for the asynchronous call to complete before you can call it again. Of course, Javascript has little support for "waiting". I was able to create a work around using a do {} while (!completed); but it causes the browser to complain mightly, ties up a lot of CPU and is just generally ugly.

Seeing that this wasn't going to work very well, I went back to the drawing board. If I actually could provide the Lat/Lng for each object, the problem would go away. It turns out that there are not very many services that want to let you do this for free using a desktop tool. If you are willing to do a web mashup you can get away with quite a bit, but the traditional languages are left out in the cold for the most part.

This is where MapQuest comes in. They have a free service (you have to register, but that's it) that will let you have access to their mapping systems. Of course, they have no interface for Delphi, but they did publish their XML specifications so it's not too hard to write one. Keep in mind that you have to comply with the MapQuest TOS. The major one for me was that you can't use the Lat/Lng addresses with any other system than theirs. That means I can't use maps.live.com to produce my maps, I have to figure out how to use MapQuest. Considering my other choices was a complete web mashup rewrite or some commercial system that was going to cost a fortune, I think I'll live within those parameters.

I've only implemented the single interface I needed so far (GeoCoding), but the other interfaces are equally easy. This code is virtually identical (in the HTTP POST) as my DHL code. I've reposted the whole thing here for those needing a complete unit but will likely factor it out for my personal use.

Enjoy!

1. This is how the classes are used. I assume you recognize that the txtXXX are just text box fields and the resolver form has a single control on it that is a list box (other than the Select / Cancel buttons).


uses mapquest_api, w_AddyResolver;

procedure TForm1.btnGeoCodeClick(Sender: tobject);
var geocode:TMapQuestGeoCoder;
    index, ix:integer;
begin
geocode:=TMapQuestGeoCoder.Create;                                               //create the geocoder object
geocode.Password:='YOUR PASS';                                                   //assign the password and client id you got from mapquest
geocode.ClientID:='YOUR ID';                                                     //this will NOT work without it!!!

geocode.Address    := edtAddress.Text;                                           //assign the address info you want to code
geocode.City       := edtCity.Text;
geocode.State      := edtState.Text;
geocode.PostalCode := edtPostalCode.Text;
geocode.Country    := edtCountry.Text;

if geocode.getFromMapQuest(true) then                                            //this will send the info to mapquest and return the results
   if geocode.ReturnedAddresses.Count>0 then                                     //if we got a response
      begin
      if geoCode.ReturnedAddresses.Count=1 then                                  //if there's only 1, no need to show a resolver
         index:=0
      else                                                                       //we got multiple responses, the user will have to resolve which addy is correct
         begin
         index:=-1;                                                              //default to failure
         for ix := 0 to geoCode.ReturnedAddresses.Count-1 do                     //for each addy
            with geoCode.ReturnedAddresses.Address[ix] do
               addyResolver.lb.Items.Add(Address+' '+City+' '+State+' '+Country); //load the listbox
         if addyResolver.ShowModal=mrOk then                                     //prompt the user, if they select one, mrOk will be set
            index:=addyResolver.lb.ItemIndex;                                    //get the addy
         if index<0 then exit;                                                   //if the user failed to pick, get out
         end;

      lblLat.Caption:=geoCode.ReturnedAddresses.Address[index].Lat;              //display the lat and long values
      lblLong.Caption:=geoCode.ReturnedAddresses.Address[index].Long;
      end;

end;

2. And this is the actual unit that does the work. BTW, the XML parser I'm using is OpenXML, a freely available, full source implementation that works for Delphi and Kylix.


unit mapquest_api;

interface

uses classes, contnrs;

type
   //base class, you must derive descendants to make use of this class
   //in order to successfully communicate with mapquest, you must attach YOUR ClientID, Password and APIKey
   //you can register for free at mapquest.com
   TMapQuestAPI = class
      private
         fClientID : string;
         fPassword : string;
         fAPIKey   : string;
         fURL      : string;
      protected
         function getXML:string; virtual; abstract;
         procedure parseResponse(response:TStringList); virtual; abstract;
      public
         constructor Create;
         function getFromMapQuest(enableLogging:boolean):boolean;

         property ClientID : string read fClientID write fClientID;
         property Password : string read fPassword write fPassword;
         property APIKey   : string read fAPIKey   write fAPIKey;
         property URL      : string read fURL;
      end;

   TReturnedAddyList=class;

   //this descendant will implement the GeoCoder functionality from mapquest
   //you must comply with mapquests terms of service, check mapquest.com to make sure
   //that works for you
   TMapQuestGeoCoder = class(TMapQuestAPI)
      private
         fAddress   : string;
         fState     : string;
         fCity      : string;
         fCountry   : string;
         fPostalCode: string;
         fCounty    : string;
         fReturnedAddresses : TReturnedAddyList;
      protected
         function getXML:string; override;
         procedure parseResponse(response:TStringList); override;
         procedure ClearReturnedAddys;
      public
         constructor Create;
         destructor Done;

         property Address:string    read fAddress    write fAddress;
         property City:string       read fCity       write fCity;
         property State:string      read fState      write fState;
         property Country:string    read fCountry    write fCountry;
         property PostalCode:string read fPostalCode write fPostalCode;
         property County:string     read fCounty     write fCounty;
         property ReturnedAddresses:TReturnedAddyList read fReturnedAddresses;
      end;

   //container class for a returned address
   TReturnedAddy = class
      public
         Address,
         City,
         State,
         PostalCode,
         Country,
         County,
         Lat,
         Long      : string;

         constructor Create(aAddress, aCity, aState, aPostalCode, aCountry, aCounty, aLat, aLong:string);
      end;

   //you can get back MULTIPLE addresses if you send in something that is generic enough
   //in that case, the COUNT will be > 1. Make sure you implement a resolver screen to handle this
   TReturnedAddyList = class(TList)
      protected
         property Items;
         property List;
      public
         function getAddy(index:integer):TReturnedAddy;
         property Address[Index:Integer]:TReturnedAddy read getAddy;
      end;


implementation

uses sysutils, dialogs, IdHttp, IdSSLOpenSSL, IdResourceStringsCore, IdLogFile, xdom_3_2;

const
   CR = #13;
   LF = #10;
   EOL = CR+LF;
   xmlEOL = EOL;

type
  // this is a descendant of TidLogFile, it will create a plain text file with
  // information about the transfer session
  TlogFile = class(TidLogFile)
  protected
    procedure LogReceivedData(const AText, AData: string); override;
    procedure LogSentData(const AText, AData: string); override;
    procedure LogStatus(const AText: string); override;
  public
    procedure LogwriteString(const AText: string); override;
    class function buildLogLine(data, prefix: string) : string;
  end;

// this ensures the output of error and debug logs are in the same format, regardless of source
class function TlogFile.buildLogLine(data, prefix: string) : string;
begin

  data := StringReplace(data, EOL, RSLogEOL, [rfReplaceAll]);
  data := StringReplace(data, CR,  RSLogCR,  [rfReplaceAll]);
  data := StringReplace(data, LF,  RSLogLF,  [rfReplaceAll]);

  result := FormatDateTime('yy/mm/dd hh:nn:ss', now) + ' ';
  if (prefix <> '') then
    result := result + prefix + ' ';
  result := result + data + EOL;
end;

// ---------------------------------------------------------------------------

procedure TlogFile.LogReceivedData(const AText, AData: string);
begin
  // ignore AText as it contains the date/time
  LogwriteString(buildLogLine(Adata, '<<'));
end;

// ---------------------------------------------------------------------------

procedure TlogFile.LogSentData(const AText, AData: string);
begin
  // ignore AText as it contains the date/time
  LogwriteString(buildLogLine(Adata, '>>'));
end;

// ---------------------------------------------------------------------------

procedure TlogFile.LogStatus(const AText: string);
begin
  LogwriteString(buildLogLine(AText, '**'));
end;

// ---------------------------------------------------------------------------

procedure TlogFile.LogwriteString(const AText: string);
begin
  // protected --> public
  inherited;
end;

// ---------------------------------------------------------------------------

{ TMapQuestAPI }

//create the class, default everything to blank
constructor TMapQuestAPI.Create;
begin
fURL:='';
fClientID:='';
fPassword:='';
fAPIKey:='';
end;

// ---------------------------------------------------------------------------

//send the http post to mapquest and return a stringlist with the results (or nil if error)
function TMapQuestAPI.getFromMapQuest(enableLogging:boolean): boolean;
const logFileName = 'http.log';
var
  plainData : TStringList;
  HTTP     : TidHTTP;
  response : TStringList;
  logFile  : TLogFile;

  //as mentioned in my article on http post for DHL, indy converts CR/LF pairs to &.
  //This causes a problem when sending xml data.
  //This routine will remove CR/LF pairs and replace them with a space
  procedure ConvertCRLFtoSpace;
  begin
    if plainData.Count > 1 then begin
      // break trailing CR&LF
      plainData.Text := StringReplace(Trim(plainData.Text), sLineBreak, ' ',[rfReplaceAll]);
    end else begin
      plainData.Text := Trim(plainData.Text);
    end;
  end;

begin
  result:=false;                                                           //by default, we get failure response in error
  plainData := TStringList.Create;                                         //init the container for xmiting the xml
  try
     plainData.Text:=getXML;                                               //get the xml from the class and put it in the container
     Http:=TidHTTP.Create(nil);                                            //create the http class
     try
        HTTP.readTimeout := 10000;                                         //setup the http timeouts and xmit types
        HTTP.ConnectTimeout := 10000;
        HTTP.Request.Contenttype := 'text/xml';
        HTTP.HTTPOptions := [];
        if enableLogging then                                              //if we are going to log the events (useful if there's a problem)
           begin
           logFile := TLogFile.Create(nil);                                //create the log file
           logFile.FileName:=logFileName;                                  //set the default log file name - NOTE: you could get fancy with this, but I normally don't use it
           logFile.Active:=true;                                           //activate the log file (creates the file and intializes everything)
           HTTP.Intercept := logFile;                                      //set the logFile to the intercept of the HTTP to capture the events
           end;
        response:=TStringList.Create;                                      //create the response container
        try
           ConvertCRLFtoSpace;                                             //make sure you strip cr/lf pairs out or the xml will be malformed
           response.Text := HTTP.Post(URL, plainData);                     //post the xml and receive the response xml
           result:=true;
           parseResponse(response);
        finally
           response.Free;
        end;
     finally
        Http.free;                                                         //free the http class
     end;
  finally
     plainData.Free;                                                       //free the container class
  end;

end;

{ TMapQuestGeoCoder }

procedure TMapQuestGeoCoder.ClearReturnedAddys;
var ix:integer;
begin
for ix := 0 to fReturnedAddresses.Count-1 do
   fReturnedAddresses.Address[ix].Free;
fReturnedAddresses.Clear;
end;

constructor TMapQuestGeoCoder.Create;
begin
fURL:='http://geocode.free.mapquest.com/mq/mqserver.dll?e=5&';            //this is the free server address. if you sign up for a different type of account, you'll want to set this to your account type
fReturnedAddresses:=TReturnedAddyList.Create;
end;

destructor TMapQuestGeoCoder.Done;
begin
ClearReturnedAddys;
fReturnedAddresses.Free;
end;

function TMapQuestGeoCoder.getXML: string;                                //returns the xml formated per the MapQuest API
const geocodeXML =
   '<?xml version="1.0" encoding="ISO-8859-1"?>'+
   '<Geocode version="1">'+

      '<Address>'+
        '<AdminArea1>%s</AdminArea1>'+
        '<AdminArea3>%s</AdminArea3>'+
        '<AdminArea5>%s</AdminArea5>'+
        '<PostalCode>%s</PostalCode>'+
        '<Street>%s</Street>'+
      '</Address>'+

      '<GeocodeOptionsCollection Count="0"/>'+

      '<Authentication Version="2">'+
        '<Password>%s</Password>'+
        '<ClientId>%s</ClientId>'+
      '</Authentication>'+
   '</Geocode>';

begin
result := Format(geocodeXML, [Country,State,City,PostalCode,Address,Password,ClientID]);  //make the request xml for geocding an address
end;

procedure TMapQuestGeoCoder.parseResponse(response: TStringList);
var
    DOC:TXMLtoDomParser;
    DOM:TDOMImplementation;
    XML:TDOMDocument;
    LocCollection:TDomNodeList;
    Geo,
    Node:TDomNode;

    //you can't trust that the xml contains ALL element names so this wraps the testing conditions up
    //to safely get the node values or a blank when not in the xml
    function SafeGetNodeValue(node:TDomNode; name:string):string;
    var DomElement:TDomElement;
    begin
       result:='';
       DomElement:=Node.GetFirstChildElement(name);
       if DomElement=nil then exit;
       if DomElement.ChildNodes=nil then exit;
       if DomElement.ChildNodes.Length=0 then exit;
       result:=DomElement.ChildNodes.Item(0).NodeValue;
    end;

    //same as the previous but designed to get a subelement (LatLng/Lat or LatLng/Lng)
    function SafeGetSubNodeValue(node:TDomNode; rootName, subName:string):string;
    var RootElement,
        SubElement:TDomElement;
    begin
       result:='';
       RootElement:=Node.GetFirstChildElement(rootName);
       if RootElement=nil then exit;
       SubElement:=RootElement.GetFirstChildElement(subName);
       if subElement=nil then exit;
       if subElement.ChildNodes=nil then exit;
       if subElement.ChildNodes.Length=0 then exit;
       result:=subElement.ChildNodes.Item(0).NodeValue;
    end;

begin
   ClearReturnedAddys;                                                                             //make sure there are no saved addys
   DOC:=TXMLtoDomParser.Create(nil);                                                               //create the xml parser
   DOM:=TDOMImplementation.Create(nil);                                                            //create the dom
   try
     DOC.DOMImpl:=DOM;                                                                             //assign the dom to the xml parser
     XML := DOC.StringtoDom(response.Text,'mapquest.xml',nil,true);                                //convert the response text to and xml document. NOTE: 'mapquest.xml' forces the correct creation of the node structures. If you pass blank, you get NO NODES
     try
        LocCollection := XML.GetElementsByTagName('LocationCollection');                           //get the location collection (group of geoaddresses 1..n)
        if LocCollection=nil then exit;

        Node:=LocCollection.Item(0);                                                               //get the top level node
        Geo:=Node.GetFirstChildElement('GeoAddress');                                              //get the first GeoAddress
        while Geo<>nil do                                                                          //while we have a GeoAddress
           begin
           fReturnedAddresses.Add(TReturnedAddy.Create(SafeGetNodeValue(Geo, 'Street'),            //addy
                                                       SafeGetNodeValue(Geo, 'AdminArea5'),        //city
                                                       SafeGetNodeValue(Geo, 'AdminArea3'),        //state
                                                       SafeGetNodeValue(Geo, 'PostalCode'),        //zip
                                                       SafeGetNodeValue(Geo, 'AdminArea1'),        //country
                                                       SafeGetNodeValue(Geo, 'AdminArea4'),        //county
                                                       SafeGetSubNodeValue(Geo, 'LatLng', 'Lat'),  //Latitude
                                                       SafeGetSubNodeValue(Geo, 'LatLng', 'Lng')   //Longitude
                                                       ));
           Geo:=Geo.NextSibling;                                                                   //get the next GeoAddress
           end;
     finally
        XML.Free;                                                                                  //free the XML document
     end;
   finally
      DOM.Free;                                                                                    //free the dom
      DOC.Free;                                                                                    //free the xml parser
   end;
end;

{ TReturnedAddy }

//create the object and initialize the data
constructor TReturnedAddy.Create(aAddress, aCity, aState, aPostalCode, aCountry, aCounty, aLat, aLong: string);
begin
Address:=aAddress;
City:=aCity;
State:=aState;
PostalCode:=aPostalCode;
Country:=aCountry;
County:=aCounty;
Lat:=aLat;
Long:=aLong;
end;

{ TReturnedAddyList }

//easy way to get access to the accumulated returned addresses
function TReturnedAddyList.getAddy(index: integer): TReturnedAddy;
begin
result:=TReturnedAddy(Items[index]);
end;

end.

Delphi in the workgroup

2008-08-12T10:34:00.000-07:00

Serge Dosyukov recently posted about the speed of the Delphi installer. While I agree that it's annoying to have to wait for the initial installation, I find that one of the largest failures in most development environments (including Delphi) is uniformity in a group setting.

For some reason, you can use products like subversion to manage your source code, but there is no product to standardize and manage your development environment. In our case, we have a handful of developers who all are working on different aspects of a large application (>1 million lines of code). We need to be able to manage the third-party add-ons and Delphi patches in a uniform way. The only method of doing this (to my knowledge) is to carefully install everything in the same paths and, usually, same order.

At one point I thought we could be clever and tried to install all add-ons to a networked drive for every workstation to minimize the amount of thrash we encounter when installing SP for the add-ons. Needless to say, that was a terrible idea. We encountered problems with locks being placed on the files which caused Delphi to either crash outright or to remove the packages that were causing the problem. It's a little disingenuous to say that it "removed the package". It prompted for the removal of the package. If we didn't, Delphi simply shut down.

I think the development environments should be modeled on a source control system. I don't mind the base install with the licensing restrictions, etc., etc. Once the base install is there though, I would like to be able to connect to a source repository that actually understands Delphi, it's add-on structures, and the registry entries needed to run, and be able to sync the changes up or down. That would let me take an add-on, install it on a test machine to make sure it works, sync it and then clone it down to the workstations. Better still, once the repository is built, let me roll back in time to something that is stable if we discover 3 months after a SP release that project XYZ just doesn't work with that SP.

If CodeGear got clever, they could setup an open API that would let the add-on vendors install to the repositories and deal with the licensing issues that everyone is concerned about. Sort of an Apple Store in an open and transparent fashion (gasp!). Obviously, you could actually set the repository system up to be a true retail presence and let the add-ons register and sell via the repository. I'm sure everyone who is using Share-It is paying a small fee anyway. Wouldn't it be a better idea to capture that fee AND make everyone's life easier at the same time?

How do you handle the group environment? The last time we looked at this problem in depth we were on D7. Is there a better way in D2007?

Indy and HTTPS Post

2008-07-21T16:05:00.000-07:00

I'm back to work on the HTTPS Post issue with DHL's ShipIt XML API. The problem I'm having is when I use Indy to post the xml via an HTTPS post, the response is that the file is malformed. I should mention that I'm using a relatively recent version of Indy 10. If I use FF (if you don't have it, check out the Poster add-on, let's you play with the HTTP functions using FF's libraries) it works fine. The first problem is that it's a little hard to see what FF is actually doing that is different from what Indy is doing. Enter Fiddler, stage right. Fiddler is an http proxy that captures that http information and lets you manipulate it or display the raw headers. So, pointing FF to the localhost proxy on port 8888, the headers I get are:



POST /apilandingtest.asp HTTP/1.1
Host: ecommerce.airborne.com
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.0.1) Gecko/2008070208 Firefox/3.0.1
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Content-Type: text/xml
Content-Length: 1509
Pragma: no-cache
Cache-Control: no-cache

Applying a TIdLogFile to the Indy HTTP IO Intercept provides Indy's headers of:



POST /apilandingtest.asp HTTP/1.0
Content-Type: multipart/form-data; boundary=--------072108153606968
Content-Length: 1662
Host: ecommerce.airborne.com:443
Accept: text/html, */*
Accept-Encoding: identity
User-Agent: Mozilla/3.0 (compatible; Indy Library)

----------072108153606968
Content-Disposition: form-data; name="Shipment"; filename="dhl.xml"
Content-Type: text/xml

Obviously, not the same. A large part of this is because I'm using the TidMultiPartFormData class to encapsulate the data. Changing over to just a plain TStringList results in headers of :



POST /apilandingtest.asp HTTP/1.0
Content-Type: text/xml
Content-Length: 1471
Host: ecommerce.airborne.com:443
Accept: text/html, */*
Accept-Encoding: identity
User-Agent: Mozilla/3.0 (compatible; Indy Library)

but DHL still won't take the file, responding again that the XML is malformed. Looking into the log a little more, it appears as if the HTTP Post converted all CR/LF pairs into the symbol &. Sure enough, in Indy's code, there is a converter that converted CR/LF to &. Stripping CR/LF pairs out of the raw XML produces a workable solution that passes DHL's IIS server and returns the correct response. Obviously, that's a bit of a poor solution since it leaves the XML mashed together and I tend to use a text editor to review my XML data more often than an XML editor. So, with a short bit of preprocess work that converts CR/LF pairs to spaces, I now have a working Indy SSL HTTPS Post routine that DHL will accept. I've posted the code below so you won't have to go digging all over creation for a sample of Indy 10's HTTPS Post. Enjoy!


const
   CR = #13;
   LF = #10;
   EOL = CR+LF;
   xmlEOL = EOL;

type
  // this is a descendant of TidLogFile, it will create a plain text file with
  // information about the transfer session
  TlogFile = class(TidLogFile)
  protected
    procedure LogReceivedData(const AText, AData: string); override;
    procedure LogSentData(const AText, AData: string); override;
    procedure LogStatus(const AText: string); override;
  public
    procedure LogWriteString(const AText: string); override;
    class function buildLogLine(data, prefix: string) : string;
  end;

// this ensures the output of error and debug logs are in the same format, regardless of source
class function TlogFile.buildLogLine(data, prefix: string) : string;
begin

  data := StringReplace(data, EOL, RSLogEOL, [rfReplaceAll]);
  data := StringReplace(data, CR,  RSLogCR,  [rfReplaceAll]);
  data := StringReplace(data, LF,  RSLogLF,  [rfReplaceAll]);

  result := FormatDateTime('yy/mm/dd hh:nn:ss', now) + ' ';
  if (prefix <> '') then
    result := result + prefix + ' ';
  result := result + data + EOL;
end;

// ---------------------------------------------------------------------------

procedure TlogFile.LogReceivedData(const AText, AData: string);
begin
  // ignore AText as it contains the date/time
  LogWriteString(buildLogLine(Adata, '<<'));
end;

// ---------------------------------------------------------------------------

procedure TlogFile.LogSentData(const AText, AData: string);
begin
  // ignore AText as it contains the date/time
  LogWriteString(buildLogLine(Adata, '>>'));
end;

// ---------------------------------------------------------------------------

procedure TlogFile.LogStatus(const AText: string);
begin
  LogWriteString(buildLogLine(AText, '**'));
end;

// ---------------------------------------------------------------------------

procedure TlogFile.LogWriteString(const AText: string);
begin
  // protected --> public
  inherited;
end;
                                                       
// ---------------------------------------------------------------------------

//this will use an https post to push the data
procedure TDHLShipment.sendFile;
const testURL   = 'https://some url that accepts data'; //replace this with the url of where you will be sending the data or send it as a param
      logFileName = 'j:\winprog\main\http.log';     //the logfile name if you use the logging
var
  plainData : TStringList;                              //this is the unencoded data
//  formData : TIdMultiPartFormDataStream;              //I've left this in just in case you want to see how the multi-part works
  HTTP   : TidHTTP;                                     //going to need the HTTP object
  SSLIO : TIdSSLIOHandlerSocketOpenSSL;                 //going to need the SSL object
  response : TStringList;                               //this will catch the response from the server
  logFile : TLogFile;                                   //this is the logfile and is optional but handy for debuging

  //this is because the plainData TStringList may contain CR/LF pairs. If so, HTTP Post will convert those pairs to & which, in my specific case, kills the data.
  procedure ConvertCRLFToSpace;
  begin
    if plainData.Count > 1 then
      // convert CR/LF pairs to spaces
      plainData.Text := StringReplace(Trim(plainData.Text), sLineBreak, ' ',[rfReplaceAll]);
  end;

begin
//  formData := TIdMultiPartFormDataStream.Create;          //if you wanted to send multi-part data, use this instead
  plainData := TStringList.Create;                          //My specific case wants stripped down headers and no multipart functionality
  try
//     formData.AddFile('Shipment', 'dhl.xml', 'text/xml'); //you can add a lot more detail about the file with the multi part data stream
     plainData.LoadFromFile('dhl.xml');                     //or you can just load up the plain file
     Http:=TidHTTP.Create(nil);                             //create the HTTP instance
     try
        SSLIO:=TIdSSLIOHandlerSocketOpenSSL.Create(nil);    //create the SSL handler
        SSLIO.SSLOptions.Method := sslvSSLv3;               //set the SSL mode
        HTTP.ReadTimeout := 10000;                          //depending on size of upload, you may need to adjust these
        HTTP.ConnectTimeout := 10000;
        HTTP.IOHandler := SSLIO;                            //assign the SSL handler to the HTTP IO Handler or you won't be able to interact with SSL servers
        HTTP.Request.Contenttype := 'text/xml';             //I'm doing XML, set the content type appropriate to your data type. If you used the multi-part, you probably don't need this
        HTTP.HTTPOptions := []; //no options
        logFile := TLogFile.Create(nil);                    //create the log file
        logFile.FileName:=logFileName;                      //set the default log file name - NOTE: you could get fancy with this, but I normally don't use it
        logFile.Active:=true;                               //activate the log file (creates the file and intializes everything)
        HTTP.IOHandler.Intercept := logFile;                //set the logFile to the intercept of the IOHandler to capture the events
        response:=TStringList.Create;                       //create the string list to catch the server's response
        try
           try
              ConvertCRLFToSpace;                           //clean up the CR/LFs so the data doesn't get funky (you may not need this)
              response.Text := HTTP.Post(testURL, plainData); //post the data to the URL and place the response in the response string list
           except on EIdOSSLCouldNotLoadSSLLibrary do         //if we have an SSL problem, this is going to be when it shows
              ShowMessage('An error occurred loading the SSL '+ //provide a little clue although, as I mentioned yesterday, you may need to change the SSL loader to get good error messages
                          'libraries. Please make sure you '+
                          'have installed the OpenSSL '+
                          'libraries from http://www.'+
                          'openssl.org/related/binaries.html.');
           end;
           response.SaveToFile('response.xml');             //save the server's response (or process it or whatever you're going to do)
           ShowMessage('Sent successfully. Response='+#13#10+response.Text); //tell the user it worked
        finally
           response.Free;                                   //free the response
        end;
     finally
        Http.free;                                          //free the http object
     end;
  finally
     plainData.Free;                                        //free the data
     //formData.Free;
  end;

end;

Indy and SSL

2008-07-20T20:27:00.000-07:00

If anyone has ever tried to get the OpenSSL libraries to work with Indy, you may have experienced a lot of pain with the process. I'm in the process of trying to develop an application for DHL's ShipIT XML Api that utilizes an HTTPS POST. The actual post code from Indy is easy

response := HTTP.Post(url, file);

but getting the SSL side to work, especial on Vista, is painful. I should disclose here that I have yet to get the HTTPS to send the data correctly via Indy, but I did finally get OpenSSL playing with Indy under Vista.

Basically, the problem stems from the fact that Indy lets you make SSL assignments that will fail at run-time and then provides very little in the way of information about the failure. This is caused by IdSSLOpenSSLHeaders.pas not having any meaningful error reporting on the OpenSSL libraries failing to load. To be clear, it does basically say "Can't load library" but that's it.

To fix it, search on SafeLoadLibrary in IdSSLOpenSSLHeaders.pas. You'll find a section that basically looks like:


  {$IFNDEF FPC}
    {$IFDEF WIN32_OR_WIN64_OR_WINCE}
  if hIdCrypto = 0 then begin
    hIdCrypto := SafeLoadLibrary(SSLCLIB_DLL_name);
  end;
  if hIdSSL = 0 then begin
    hIdSSL := SafeLoadLibrary(SSL_DLL_name);
  end else begin
    Exit;
  end;
    {$ENDIF}
 {$ENDIF}

What is happening here is that hIdCrypto and hIdSSL are attempting to load the OpenSSL DLLs. If this fails, there is no trap to capture the error. I recommend changing the loading to:


  {$IFNDEF FPC}
    {$IFDEF WIN32_OR_WIN64_OR_WINCE}
  if hIdCrypto = 0 then begin
    hIdCrypto := SafeLoadLibrary(SSLCLIB_DLL_name);
    if hIDCrypto<1 then
       Windows.MessageBox(0, pansichar('An error occured loading '+SSLCLIB_DLL_name+#13#10+ErrorDefinition(getLastError)), 'Error',
                          MB_TASKMODAL+MB_SETFOREGROUND+MB_TOPMOST);
  end;
  if hIdSSL = 0 then begin
    hIdSSL := SafeLoadLibrary(SSL_DLL_name);
    if hIDSSL<1 then
       Windows.MessageBox(0, pansichar('An error occured loading '+SSL_DLL_name+#13#10+ErrorDefinition(getLastError)), 'Error',
                          MB_TASKMODAL+MB_SETFOREGROUND+MB_TOPMOST);
  end else begin
    Exit;
  end;
    {$ENDIF}
 {$ENDIF}

Oh, you'll also need my little cheat routine called ErrorDefinition. It just takes the last Windows error (which you pass, otherwise you couldn't save it if you had some type of log that kept track of the error number and the error message) and converts it to a string. For some reason, I find error #14401 not terribly useful.


function ErrorDefinition(x:longint):string;
var Buffer : array[0..255] of Char;

    function TrimWhite(const S : String) : String;
    var
      i : Integer;
    begin
      Result := S;
      for i := length(Result) downto 1 do
        if Result[i] < ' ' then
          Result[i] := ' ';
      i := pos('  ',Result);
      while i <> 0 do begin
        delete(Result,i,1);
        i := pos('  ',Result);
      end;
    end;

begin
   result:='';
   //ask for error message from OS
   if FormatMessage(FORMAT_MESSAGE_FROM_SYSTEM,nil,abs(x),0,
                    Buffer,255,nil) > 0 then
      Result := TrimWhite(StrPas(Buffer));
end;

Some of you may want to change the MessageBox function over to a raise Exception.Create... but I'll leave that to your discretion. For my purposes, I was just trying to get something that gave more details than "Unable to load". Of course, the next problem is that the error message I got was really unhelpful. In my case, the message says that there is a problem with the SideBySide configuration and to check the event viewer for the application log. Since I have no idea what a SideBySide configuration is, I check the application log to discover this message


 "Activation context generation failed for "j:\winprog\main\ssleay32.dll". Dependent Assembly Microsoft.VC90.CRT,processorArchitecture="x86",publicKeyToken="1fc8b3b9a1e18e3b",type="win32",version="9.0.21022.8" could not be found. Please use sxstrace.exe for detailed diagnosis."

Now I'm really confused. I've just installed the OpenSSL libraries via the installer. Just to make sure, I check that the SSL DLLs actually are present in the system32 directory. Just to make sure, I copy them local and still have the problem. I then download the special OpenSSL libraries that are supposed to work better with Indy and have the same results. BTW, I did use the sxstrace tool which is (remarkably) easy to use. The problem is that it points me into the SideBySide directory (c:\windows\winsxs) which doesn't look like it's designed to be edit by people. For those interested, the sxstrace tool output looks like:


"Begin Activation Context Generation.
Input Parameter:
 Flags = 0
 ProcessorArchitecture = x86
 CultureFallBacks = en-US;en
 ManifestPath = j:\winprog\main\libeay32.dll
 AssemblyDirectory = j:\winprog\main\
 Application Config File =
-----------------
INFO: Parsing Manifest File j:\winprog\main\libeay32.dll.
 INFO: Manifest Definition Identity is (null).
 INFO: Reference: Microsoft.VC90.CRT,processorArchitecture="x86",publicKeyToken="1fc8b3b9a1e18e3b",type="win32",version="9.0.21022.8"
INFO: Resolving reference Microsoft.VC90.CRT,processorArchitecture="x86",publicKeyToken="1fc8b3b9a1e18e3b",type="win32",version="9.0.21022.8".
 INFO: Resolving reference for ProcessorArchitecture x86.
  INFO: Resolving reference for culture Neutral.
   INFO: Applying Binding Policy.
    INFO: No publisher policy found.
    INFO: No binding policy redirect found.
   INFO: Begin assembly probing.
    INFO: Did not find the assembly in WinSxS.
    INFO: Attempt to probe manifest at C:\Windows\assembly\GAC_32\Microsoft.VC90.CRT\9.0.21022.8__1fc8b3b9a1e18e3b\Microsoft.VC90.CRT.DLL.
    INFO: Attempt to probe manifest at j:\winprog\main\Microsoft.VC90.CRT.DLL.
    INFO: Attempt to probe manifest at j:\winprog\main\Microsoft.VC90.CRT.MANIFEST.
    INFO: Attempt to probe manifest at j:\winprog\main\Microsoft.VC90.CRT\Microsoft.VC90.CRT.DLL.
    INFO: Attempt to probe manifest at j:\winprog\main\Microsoft.VC90.CRT\Microsoft.VC90.CRT.MANIFEST.
    INFO: Did not find manifest for culture Neutral.
   INFO: End assembly probing.
 ERROR: Cannot resolve reference Microsoft.VC90.CRT,processorArchitecture="x86",publicKeyToken="1fc8b3b9a1e18e3b",type="win32",version="9.0.21022.8".
ERROR: Activation Context generation failed.
End Activation Context Generation."

At any rate, Google of course tells me that I'm missing the MS VC 9.0 RTL. This is surprising since I have MS VS 2008 installed on this machine but some digging in those folders doesn't turn up the VC90 DLLs (which, just to be fun, are actually named msvc?90.dll). At any rate, I found a link to them on Google Code, download them and put them AND the manifest file in the system32. For those of you who read the sxstrace dump better than I did, I'm sure you won't be surprised to find that this didn't do anything to fix the problem. I then copy the DLL files locally. Still nada. I finally get a clue and copy the DLLs AND the manifest file locally and voila, I can now load the OpenSSL libraries. Of course, I could have put the files and manifest in C:\Windows\assembly\GAC_32\Microsoft.VC90.CRT\9.0.21022.8__1fc8b3b9a1e18e3b\Microsoft.VC90.CRT.DLL (I think), but that folder doesn't really look like it should be adjusted via the command prompt AND I'm probably going to switch away from Indy for this project since the chances of getting this successfully deployed to hundreds of machines just went down a couple of notches.

At any rate, I still haven't figured out the HTTPS Post issue yet. I can post the XML to DHL fine via FF, but the Indy code is corrupting the XML somehow. I'll post again once I get that part figured out and provide the sample code that is so lacking from the Indy project. Don't get me wrong, Indy rocks for what it does, but they could REALLY use a huge set of "for instance" samples. This blog gets 70 hits or so every week just from people looking for hints on the Indy SMTP components that I posted a while back.

Oh, and WHEN did we decided "Unable to load DLL xyz" should be renamed "SideBySide Configuration"? I don't know about you, but the former is much more descriptive than the later.

Embarcadero and the future

2008-05-07T14:17:00.000-07:00

For everyone who isn't hiding under a rock, Embarcadero has agreed to acquire CodeGear from Borland. I'm sure all of us who have been annoyed with Borland for the last 5 years are relieved that the other shoe has finally dropped.

Why 5 years you ask? Let's call a spade, a spade, and say the last really good release of Delphi was Delphi 7 until Delphi 2007 hit the scenes. That comprises a 5 year window where Delphi basically floundered around trying to be things that the core audience really didn't care about in a way that hurt sales and led everyone to assume that there would not really BE another good version of Delphi... ever. Fortunately, Delphi 2007 came out and changed that. Not right off the bat, it did take a service pack (or two) to get things flowing relatively smoothly, but I think everyone pretty much agrees that 2007 is the version to beat now.

The biggest question is probably, "What's the future of Delphi?" Not in a "Delphi sucks" sort of way. I love Delphi. It's my preferred language for any and everything. Even where it isn't quite right, I still try and cram it in. My company even wrote it's own Delphi-script library. If that isn't love, I don't know what is. What I mean by the question is really two parts : first, where are the new Delphi developers coming from and secondly, where is Delphi going?

New developers are the life blood of a language. Once the language stops attracting new developers, it's pretty much dead on the vine since everyone who is currently using it will, eventually, retire and/or move on. The positioning of Delphi in Eastern European schools is a great first step to preventing this problem. I actually think that a stronger academic program, up to and including giving the software away to academics, makes a lot of sense for both Embarcadero and the community as a whole.

For the second question, I realize that anyone can point to the roadmap and say, "See, this is where we're going"; however, I think that the roadmap is extremely limited and doesn't discuss, in detail, the types of things that the language really needs. Part of that is, perhaps, due to the formerly public nature of CodeGear. The remainder is that the language has traditionally been controlled by just one company. Not Pascal the language, but the most widely used derivative of Pascal known as Delphi.

It would be interesting if Embarcadero would split the compiler (bpcc) off as an open-source project and build the libraries and IDE on top of it. That appears to be a winning strategy for Java (ala JBuilder), why not Delphi? In fact, why have multiple different IDE's at all? Why not leverage Eclipse (again, ala JBuilder), and cut a lot of development time and effort out. Isn't code reuse a good thing?

Everyone always complains about open-source software because "there is no money" in it. I'm not suggesting that anything besides the compiler needs to be opened up. When I buy Delphi, I'm not really trying to gain access to the compiler. What I am interested in is the RAD IDE and the libraries (value-added) that CodeGear makes available to me. In fact, FPC as the compiler might be an interesting revision to bpcc. It covers the majority of the goals that CodeGear has on the roadmap (64-bit, multi-CPU, multi-OS) and is close to being Delphi-compatible. Regardless of open-sourcing bpcc or using fpc, an open compiler would both permit other developers to correct issues in the compiler and reassure developers that the tool they've chosen to put millions of lines of code into isn't going to die.

Additionally, the language is being threatened with further fragmentation as the multi-core (or parallel) extensions are being added. If you look at the work RemObjects has done with Oxygene, you can see that the ideas they have advanced (and modifications to the core language), are both useful and straight-forward. CodeGear hasn't, to my knowledge, made public their vision of what parallel extensions they are going to offer. Allen Bauer has blogged about some of the issues, but not in the kind of depth that RemObjects has shown. Parallel extensions is a core competency that could extended Delphi to be the best option for the multi-core CPU's that we are all using and create interest in the wider development community. With an open-sourced compiler, we could have multiple contributors working to build the core language without causing the fragmentation.

To further compound the issue, Delphi's fabled RAD environment should have been brought to the Web years ago in a way that makes web programming (and I'm not referring to html here) easy for everyone. For an excellent example of what I'm talking about, look at Morfik. They have a RAD environment that supports most of the Delphi-syntax (and most of the Delphi IDE functions too) that lets developers develop applications using familiar paradigms. Isn't that the whole point of buying tools? Not to spend 12 months retraining, but to buy something that makes the transition to a new environment easier.

Finally, the days of Windows being the only operating system to develop for are either already over or nearing an end. If you think this is a poor prediction, consider that Microsoft has recently released software to manage Linux servers. Between Mac OS X and the various OSS (mostly *nix) platforms, the world of homogeneous software stacks is on the way out. This doesn't even consider the tiny mobile world that has developed. Palm has sold over 1 million Centro's running Palm OS. Of that market, the current incarnation of Delphi can tap exactly 0%.

Hopefully, Embarcadero will bring vision and funding to Delphi (and all of CodeGears products) in a way that we haven't seen in far too long.

Detecting Virtual PC

2008-04-22T14:35:00.001-07:00

Adding to my previous post on detecting virtual environments, here's the code for detecting Virtual PC. Note that it's a conversion from CodeProject, the original author is here. I also didn't write the conversion, I'm simply accumulating the VMM detection code here. Original credit for the conversion goes to Dennis Pasamore who did the bulk of the conversion work with some assistance from Avatar Zonderatau.

Code:


function TForm1.IsRunningVirtualPC: boolean;
asm
  push ebp;
  mov ebp, esp;

  mov ecx, offset @exception_handler;

  push ebx;
  push ecx;

  push dword ptr fs:[0];
  mov dword ptr fs:[0], esp;

  mov ebx, 0; // Flag
  mov eax, 1; // VPC function number

  // call VPC
  db $0F, $3F, $07, $0B

  mov eax, dword ptr ss:[esp];
  mov dword ptr fs:[0], eax;

  add esp, 8;

  test ebx, ebx;

  setz al;

  lea esp, dword ptr ss:[ebp-4];
  mov ebx, dword ptr ss:[esp];
  mov ebp, dword ptr ss:[esp+4];

  add esp, 8;

  jmp @ret1;

@exception_handler:
  mov ecx, [esp+0Ch];
  mov dword ptr [ecx+0A4h], -1; // EBX = -1 ->; not running, ebx = 0 -> running
  add dword ptr [ecx+0B8h], 4;  // ->; skip past the call to VPC
  xor eax, eax;                 // exception is handled

@ret1:
end;

Detecting a virtualized environment

2008-04-21T13:16:00.000-07:00

CubicDesign on delphi-talk.elists.org recently asked the question: "How do I know/detect if my software is running under Windows [or a virtual environment]?" Well, it turns out that it's a lot harder to tell than you would think. Apparently, the VM (VMware, Xen, Wine, etc.) doesn't really want you to be able to do this. At least not easily.

For VMware, there is a decent little C routine called jerry.c that does the trick. Jerry actually uses a simple communication channel that VMware left open. It's not 100% foolproof since the admin can change the channel, but that's not likely going to happen unless the system is specifically designed to be a honeypot. If you're running on a honeypot and still have a legitimate reason for detection, you could look at the much more complex scoopy implementation which inspects how the system is actually virtualized using the SIDT CPU instruction instead of a communication channel. Another reference (red pill) is here.

For Wine, there is a special dll entry called wine_nt_to_unix_file_name that they added to NTDLL.DLL. If you use LoadLibrary and GetProcAddress, you can determine if you are under Wine simply by the entry's presence or absence. Of course, if you specifically want to know what version of Wine you're on, you have to use NTDLL_wine_get_version but that function is only available in the bleeding edge version of wine. As an interesting side note, the Jerry test above actually returns true for VMware workstation under Wine. If you really have to know Wine vs VMware, make sure you test Wine first, otherwise you'll get a false positive.

For Xen, the most reliable method appears to be to use the WMI interface and get the Bios manufacturer's information. If it's Xen, then you are under some version of Xen. Since I don't have a Xen environment setup currently, I couldn't test this information. I am relying on information published here to make this assertion. At some point, I'll setup a Xen server and play with it.

For true VM's like Xen and Vmware, you can also test against the mac address. VMware uses 00-05-69 as the lead and Xen uses 00-16-3E. To see a list of all registered Mac addresses, go here. Keep in mind, that the VMM supports changing the MAC address so this is also easily defeated.

At any rate, here's some sample code that uses the described methods to detect VMware and Wine. Enjoy!

VMware:


//credit to chitchat@lycos.jp who discovered this method and
//checkvm + IDA Pro ;)
//converted from C source posted on www.trapkit.de
procedure TForm1.btnJerryClick(Sender: TObject);
var a, b:cardinal;
begin
a:=0;
try
   asm
      push eax
      push ebx
      push ecx
      push edx

      mov eax, 'VMXh'
      mov ecx, 0Ah
      mov dx, 'VX'

      in eax, dx

      mov a, ebx
      mov b, ecx

      pop edx
      pop ecx
      pop ebx
      pop eax
   end;
except on E:Exception do ShowMessage(E.Message);
end;
if a=$564D5868 then
   begin
   ShowMessage('In VMware');
   case b of
   1  : ShowMessage('Express');
   2  : ShowMessage('ESX');
   3  : ShowMessage('GSX');
   4  : ShowMessage('Workstation');
   else ShowMessage('Unknown version')
   end;
   end
else
   ShowMessage('Native system');
end;

Wine:


function TForm1.IsRunningWine: boolean;
var hnd:THandle;
    wine_get_version: function : pchar; {$IFDEF Win32} stdcall; {$ENDIF}
    wine_unix2fn: procedure (p1:pointer; p2:pointer); {$IFDEF Win32} stdcall; {$ENDIF}
begin
result:=false;
hnd:=LoadLibrary('ntdll.dll');
if hnd>32 then
   begin
   wine_get_version:= GetProcAddress(hnd, 'wine_get_version');
   wine_unix2fn:= GetProcAddress(hnd, 'wine_nt_to_unix_file_name');
   if assigned(wine_get_version) or assigned(wine_unix2fn) then
      result:=true;
   FreeLibrary(hnd);
   end;

end;

Calendar Conversions

2008-04-15T12:15:00.000-07:00

Has anyone tried to convert a Western-style calendar to either the Islamic or Hebrew calendar system? I've spent a bunch of time looking for a Pascal unit to do just that so I could include Muslim and Jewish holidays in my holiday class but haven't found any great resources. I finally found some C/C++ code that did the work and converted it over to Pascal. Keeping in mind that this is still a new conversion and that I didn't write the actual conversion logic, I'm publishing the code to help other would-be calendar converters. All of the conversion algorithms (and credit for them) are from Calendrical Calculations by Nachum Dershowitz and Edward Reingold.

I added the Day and Month name strings from various Google searches on things like Hebrew Month Names and Islamic Month Names, etc. I've compared the resulting output to both published calendars and to other conversion tools that do not provide their source. So far, I haven't seen any discrepancies other than some naming conventions. Apparently, Hebrew and Arab names are "open to interpretation" when you write them in Western languages. Feel free to change them to your particular dialect. Actually, I'd like to know what the differences are/could be/should be if someone wants to enlighten me. I don't speak either language and am relying on other translations to make this work.

Enjoy the code.


unit calconv;

interface
// The following Pascal code is translated from the Lisp code in
// ``Calendrical Calculations'' by Nachum Dershowitz and Edward M. Reingold,
// Software---Practice & Experience, vol. 20, no. 9 (September, 1990),
// pp. 899--928.

// This code is in the public domain, but any use of it
// should acknowledge its source.

// Classes GregorianDate, JulianDate, IslamicDate, and HebrewDate

// Translated from C++ / Lisp sources as published by Edward Reingold
// at http://emr.cs.iit.edu/~reingold/calendars/shtml

// Pascal translation by Marshall Fryman, 2008

uses sysutils;

type

   TBaseDate = class
      private
         fYear:integer;
         fMonth:integer;
         fDay:integer;
      public
         class function cf_CalDateAsAbsDate(m,d,y:integer):integer; virtual; abstract;
         class function DayName(d:integer):string; virtual; abstract;
         class function LastDayOfMonth(m,y:integer):integer; virtual; abstract;
         function CalDateAsAbsoluteDate:integer;
         procedure AbsoluteDateToCalDate(absdate:integer); virtual; abstract;
         property Year:integer read fYear;
         property Month:integer read fMonth;
         property Day:integer read fDay;
      end;

   TGregorianDate = class(TBaseDate)
//      inherited from base
//        fYear   : integer;   // 1...
//        fMonth  : integer;   // 1 == January, ..., 12 == December
//        fDay    : integer;   // 1..LastDayOfMonth(fMonth, fYear)
      public
        constructor Create(m,d,y:integer);   overload; { fMonth = m; fDay = d; fYear = y; }
        constructor Create(DT:TDateTime);    overload;
        constructor Create(absdate:integer); overload;
        function DateAsText:string;
        procedure AbsoluteDateToCalDate(absdate:integer); override;
        class function cf_CalDateAsAbsDate(m,d,y:integer):integer; override;
        class function DayName(d:integer):string; override;
        class function MonthName(m:integer):string;
        class function LastDayOfMonth(m,y:integer):integer; override;
      end;

   TJulianDate = class(TBaseDate)
//      inherited from base
//        int fYear;   // 1...
//        int fMonth;  // 1 == January, ..., 12 == December
//        int fDay;    // 1..LastDayOfMonth(fMonth, fYear)
      public
        constructor Create(m,d,y:integer);   overload; { fMonth = m; fDay = d; fYear = y; }
        constructor Create(DT:TDateTime);    overload;
        constructor Create(absdate:integer); overload;
        procedure AbsoluteDateToCalDate(absdate:integer); override;
        class function cf_CalDateAsAbsDate(m,d,y:integer):integer; override;
        class function DayName(d:integer):string; override;
        class function MonthName(m:integer):string;
        class function LastDayOfMonth(m,y:integer):integer; override;
      end;

   TIslamicDate = class(TBaseDate)
//      inherited from base
//        int fYear;   // 1...
//        int fMonth;  // 1..12 (12 in a common Year)
//        int fDay;    // 1..LastDayOfMonth(fMonth,fYear)
      public
        constructor Create(m,d,y:integer);   overload; { fMonth=m; fDay=d; fYear =y; }
        constructor Create(DT:TDateTime);    overload;
        constructor Create(absdate:integer); overload;
        function DateAsText:string;
        procedure AbsoluteDateToCalDate(absdate:integer); override;
        class function cf_CalDateAsAbsDate(m,d,y:integer):integer; override;
        class function DayName(d:integer):string; override;
        class function MonthName(m:integer):string;
        class function LastDayOfMonth(m,y:integer):integer; override;
        class function IsLeapYear(y:integer):boolean;
   end;

   THebrewDate = class(TBaseDate)
//      inherited from base
//        int fYear;   // 1...
//        int fMonth;  // 1..LastMonthOfYear(fYear)
//        int fDay;    // 1..LastDayOfMonth(fMonth, fYear)
        constructor Create(m,d,y:integer);   overload; { fMonth=m; fDay=d; fYear =y; }
        constructor Create(DT:TDateTime);    overload;
        constructor Create(absdate:integer); overload;
        function DateAsText:string;
        procedure AbsoluteDateToCalDate(absdate:integer); override;
        class function cf_CalDateAsAbsDate(m,d,y:integer):integer; override;
        class function DayName(d:integer):string; override;
        class function MonthName(m,y:integer):string;
        class function LastDayOfMonth(m,y:integer):integer; override;
        class function IsLeapYear(y:integer):boolean;
        class function LastMonthOfYear(y:integer):integer;
        class function ShortKislev(y:integer):boolean;
        class function LongHeshvan(y:integer):boolean;
        class function DaysInYear(y:integer):integer;
        class function HebrewCalendarElapsedDays(y:integer):integer;
   end;

implementation

// Absolute dates

// "Absolute date" means the number of days elapsed since the Gregorian date
// Sunday, December 31, 1 BC. (Since there was no fYear 0, the fYear following
// 1 BC is 1 AD.) Thus the Gregorian date January 1, 1 AD is absolute date
// number 1.

const
   JulianEpoch = -2; // Absolute date of start of Julian calendar
   IslamicEpoch = 227014; // Absolute date of start of Islamic calendar
   HebrewEpoch = -1373429; // Absolute date of start of Hebrew calendar



// Absolute date of the x-fDay on or before absolute date d.
// x=0 means Sunday, x=1 means Monday, and so on.
function XdayOnOrBefore(d,x:integer):integer;
begin
  result:=(d - ((d - x) mod 7));
end;



{ TGregorianDate }

constructor TGregorianDate.Create(m,d,y:integer);
begin
fMonth :=m;
fDay   :=d;
fYear  :=y;
end;

// Computes the Gregorian date from the absolute date
constructor TGregorianDate.Create(absdate:integer);
begin
AbsoluteDateToCalDate(absdate);
end;


// Computes the absolute date from the Gregorian date.
constructor TGregorianDate.Create(DT: TDateTime);
var y,m,d:word;
begin
DecodeDate(DT,y,m,d);
fYear:=y;
fMonth:=m;
fDay:=d;
end;

procedure TGregorianDate.AbsoluteDateToCalDate(absdate: integer);
begin
    // Search forward fYear by fYear from approximate fYear
    fYear := absdate div 366;
    while (absdate >= TGregorianDate.cf_CalDateAsAbsDate(1,1,fYear+1)) do
      inc(fYear);
    // Search forward fMonth by fMonth from January
    fMonth := 1;
    while (absdate > TGregorianDate.cf_CalDateAsAbsDate(fMonth, LastDayOfMonth(fMonth,fYear), fYear)) do
      inc(fMonth);
    fDay := absdate - TGregorianDate.cf_CalDateAsAbsDate(fMonth,1,fYear) + 1;
end;

class function TGregorianDate.cf_CalDateAsAbsDate(m,d,y:integer):integer;
var iD,iM:integer;
begin
    iD := d;                                         // days this fMonth
    for iM := m - 1 downto 1 do                      // days in prior months this fYear
      iD := iD + LastDayOfMonth(iM, y);
    result:=(iD                                      // days this fYear
       + 365 * (y - 1)                               // days in previous years ignoring leap days
       + (y - 1) div 4                               // Julian leap days before this fYear...
       - (y - 1) div 100                             // ...minus prior century years...
       + (y - 1) div 400);                           // ...plus prior years divisible by 400
end;

function TGregorianDate.DateAsText: string;
begin
result := DayName(CalDateAsAbsoluteDate mod 7)+', '+MonthName(Month)+' '+IntToStr(Day)+', '+IntToStr(Year);
end;

class function TGregorianDate.DayName(d: integer): string;
begin
if d>6 then d:=d mod 7;
case d of
0 : result:='Sunday';
1 : result:='Monday';
2 : result:='Tuesday';
3 : result:='Wednesday';
4 : result:='Thursday';
5 : result:='Friday';
6 : result:='Saturday';
else
   result:='Invalid Day';
end;
end;

// Compute the last date of the fMonth for the Gregorian calendar.
class function TGregorianDate.LastDayOfMonth(m, y: integer): integer;
begin
  result:=31;
  case m of
   2: if ((((y mod 4) = 0) and ((y mod 100) <>0)) OR ((y mod 400) = 0)) then
         result:= 29
      else
         result:= 28;
   4,
   6,
   9,
  11: result := 30;
  end;
end;

class function TGregorianDate.MonthName(m: integer): string;
begin
case m of
1  : result := 'January';
2  : result := 'February';
3  : result := 'March';
4  : result := 'April';
5  : result := 'May';
6  : result := 'June';
7  : result := 'July';
8  : result := 'August';
9  : result := 'September';
10 : result := 'October';
11 : result := 'November';
12 : result := 'December';
else
   result:='Invalid Month';
end;
end;

{ TJulianDate }

constructor TJulianDate.Create(m, d, y: integer);
begin
fmonth := m;
fday   := d;
fyear  := y;
end;

// Computes the Julian date from the absolute date.
constructor TJulianDate.Create(absdate: integer);
begin
AbsoluteDateToCalDate(absdate);
end;

// Computes the absolute date from the Julian date.
constructor TJulianDate.Create(DT: TDateTime);
var gd:TGregorianDate;
begin
gd:=TGregorianDate.Create(DT);
AbsoluteDateToCalDate(gd.CalDateAsAbsoluteDate);
gd.Free;
end;

procedure TJulianDate.AbsoluteDateToCalDate(absdate: integer);
begin
    // Search forward fYear by fYear from approximate fYear
    fYear := (absdate + JulianEpoch) div 366;
    while (absdate >= TJulianDate.cf_CalDateAsAbsDate(1,1,fYear+1)) do
      inc(fYear);
    // Search forward fMonth by fMonth from January
    fMonth := 1;
    while (absdate > TJulianDate.cf_CalDateAsAbsDate(fMonth, LastDayOfMonth(fMonth,fYear), fYear)) do
      inc(fMonth);
    fDay := absdate - TJulianDate.cf_CalDateAsAbsDate(fMonth,1,fYear) + 1;
end;

class function TJulianDate.cf_CalDateAsAbsDate(m, d, y: integer): integer;
var iD, iM:integer;
begin
    iD:=d;                            // days this Month
    for iM:=m-1 downto 1 do           // days in prior months this Year
      iD := iD + LastDayOfMonth(m, y);
    result := (iD                     // days this fYear
           + 365 * (y - 1)            // days in previous years ignoring leap days
           + (y - 1) div 4            // leap days before this Year...
           + JulianEpoch);            // days elapsed before absolute date 1

end;

class function TJulianDate.DayName(d: integer): string;
begin
if d>6 then d:=d mod 7;

case d of
0 : result:='Sunday';
1 : result:='Monday';
2 : result:='Tuesday';
3 : result:='Wednesday';
4 : result:='Thursday';
5 : result:='Friday';
6 : result:='Saturday';
else
   result:='Invalid Day';
end;
end;

// Compute the last date of the fMonth for the Julian calendar.
class function TJulianDate.LastDayOfMonth(m, y: integer): integer;
begin
  case m of
  2: if ((y mod 4) = 0) then
       result := 29
     else
       result := 28;
   4,
   6,
   9,
   11: result := 30;
  else result := 31;
  end; {of case}
end;

class function TJulianDate.MonthName(m: integer): string;
begin
case m of
1  : result := 'January';
2  : result := 'February';
3  : result := 'March';
4  : result := 'April';
5  : result := 'May';
6  : result := 'June';
7  : result := 'July';
8  : result := 'August';
9  : result := 'September';
10 : result := 'October';
11 : result := 'November';
12 : result := 'December';
else
   result:='Invalid Month';
end;
end;

{ TIslamicDate }

constructor TIslamicDate.Create(m, d, y: integer);
begin
fmonth := m;
fday   := d;
fyear  := y;
end;

// Computes the Islamic date from the absolute date.
constructor TIslamicDate.Create(absdate: integer);
begin
AbsoluteDateToCalDate(absdate);
end;

procedure TIslamicDate.AbsoluteDateToCalDate(absdate: integer);
begin
    if (absdate <= IslamicEpoch) then // Date is pre-Islamic
      begin
      fMonth := 0;
      fDay   := 0;
      fYear  := 0;
      end
    else
      begin
      // Search forward fYear by fYear from approximate fYear
      fYear := (absdate - IslamicEpoch) div 355;
      while (absdate >= TIslamicDate.cf_CalDateAsAbsDate(1,1,fYear+1)) do
        inc(fYear);
      // Search forward fMonth by fMonth from Muharram
      fMonth := 1;
      while (absdate > TIslamicDate.cf_CalDateAsAbsDate(fMonth, LastDayOfMonth(fMonth,fYear), fYear)) do
        inc(fMonth);
      fDay := absdate - TIslamicDate.cf_CalDateAsAbsDate(fMonth,1,fYear) + 1;
      end;
end;

class function TIslamicDate.cf_CalDateAsAbsDate(m, d, y: integer): integer;
begin
    result := (d                      // days so far this fMonth
             + 29 * (m - 1)           // days so far...
             + m div 2                //            ...this fYear
             + 354 * (y - 1)          // non-leap days in prior years
             + (3 + (11 * y)) div 30  // leap days in prior years
             + IslamicEpoch);         // days before start of calendar

end;

constructor TIslamicDate.Create(DT: TDateTime);
var gd:TGregorianDate;
begin
gd:=TGregorianDate.Create(DT);
AbsoluteDateToCalDate(gd.CalDateAsAbsoluteDate);
gd.Free;
end;

function TIslamicDate.DateAsText: string;
begin
result := DayName(Day)+', '+MonthName(Month)+' '+IntToStr(Day)+', '+IntToStr(Year);
end;

class function TIslamicDate.DayName(d: integer): string;
begin
if d>6 then d:=d mod 7;
case d of
0 : result := 'As-Sabt';
1 : result := 'Al-Ahad';
2 : result := 'Al-`ithnayn';
3 : result := 'Ath-Thulatha';
4 : result := 'Al-`Arba`aa';
5 : result := 'Al-Khamees';
6 : result := 'Al-Jum`ah';
else result:='Invalid Day';
end;
end;

// True if fYear is an Islamic leap fYear
class function TIslamicDate.IsLeapYear(y: integer): boolean;
begin
  result:=((((11 * y) + 14) mod 30) < 11);
end;

// Last Day in fMonth during fYear on the Islamic calendar.
class function TIslamicDate.LastDayOfMonth(m, y: integer): integer;
begin
  if (((m mod 2) = 1) OR ((m = 12) and IsLeapYear(y))) then
    result:=30
  else
    result:=29;
end;

class function TIslamicDate.MonthName(m: integer): string;
begin
case m of
 1: result := 'Muharram';
 2: result := 'Safar';
 3: result := 'Rabi`al-Awwal';
 4: result := 'Rabi`ath-Thani';
 5: result := 'Jumada l-Ula';
 6: result := 'Jumada t=Tania';
 7: result := 'Rajab';
 8: result := 'Sha`ban';
 9: result := 'Ramadan';
10: result := 'Shawwal';
11: result := 'Dhul `l-Qa`da';
12: result := 'Dhul `l-Hijja';
else result:='Invalid Month';
end;
end;

{ THebrewDate }

constructor THebrewDate.Create(m, d, y: integer);
begin
fmonth := m;
fday   := d;
fyear  := y;
end;

// Computes the Hebrew date from the absolute date.
constructor THebrewDate.Create(absdate: integer);
begin
AbsoluteDateToCalDate(absdate);
end;

procedure THebrewDate.AbsoluteDateToCalDate(absdate: integer);
begin
    fYear := (absdate + HebrewEpoch) div 366; // Approximation from below.
    // Search forward for fYear from the approximation.
    while (absdate >= THebrewDate.cf_CalDateAsAbsDate(7,1,fYear + 1)) do
      inc(fYear);
    // Search forward for month from either Tishri or Nisan.
    if (absdate < THebrewDate.cf_CalDateAsAbsDate(1, 1, fYear)) then
      fMonth := 7  //  Start at Tishri
    else
      fMonth := 1;  //  Start at Nisan
    while (absdate > THebrewDate.cf_CalDateAsAbsDate(fMonth, (LastDayOfMonth(fMonth,fYear)), fYear)) do
      inc(fMonth);
    // Calculate the fDay by subtraction.
    fDay := absdate - THebrewDate.cf_CalDateAsAbsDate(fMonth, 1, fYear) + 1;
end;

class function THebrewDate.cf_CalDateAsAbsDate(m, d, y: integer): integer;
var DayInYear:integer;
    iM:integer;
begin
    DayInYear := d; // Days so far this Month.
    if (m < 7) then // Before Tishri, so add days in prior months
      begin             // this Year before and after Nisan.
      iM := 7;
      while (iM <= (LastMonthOfYear(y))) do
        begin
        DayInYear := DayInYear + LastDayOfMonth(iM, y);
        inc(iM);
        end;
      iM := 1;
      while (iM < m) do
        begin
        DayInYear := DayInYear + LastDayOfMonth(iM, y);
        inc(iM);
        end;
      end
    else  // Add days in prior months this Year
      begin
      iM := 7;
      while (iM < m) do
        begin
        DayInYear := DayInYear + LastDayOfMonth(iM, y);
        inc(iM);
        end;
      end;

    result := (DayInYear +
              (HebrewCalendarElapsedDays(y)// Days in prior years.
               + HebrewEpoch));            // Days elapsed before absolute date 1.
end;

constructor THebrewDate.Create(DT: TDateTime);
var gd:TGregorianDate;
begin
gd:=TGregorianDate.Create(DT);
AbsoluteDateToCalDate(gd.CalDateAsAbsoluteDate);
gd.Free;
end;

function THebrewDate.DateAsText: string;
begin
result := DayName(Day)+', '+MonthName(Month,Year)+' '+IntToStr(Day)+', '+IntToStr(Year);
end;

class function THebrewDate.DayName(d: integer): string;
begin
if d>6 then d:=d mod 7;
case d of
0 :  result := 'Yom Ree-Shoun';
1 :  result := 'Yom She-Nee';
2 :  result := 'Yom Shelee-She';
3 :  result := 'Yom Re-Ve-ee';
4 :  result := 'Yom Hah-Mee-Shee';
5 :  result := 'Yom Shee-Shee';
6 :  result := 'Yom Sha-Bat';
else result := 'Invalid Day';
end;
end;

// Number of days in Hebrew Year.
class function THebrewDate.DaysInYear(y: integer): integer;
begin
  result := ((HebrewCalendarElapsedDays(y + 1)) -
            (HebrewCalendarElapsedDays(y)));
end;

// Number of days elapsed from the Sunday prior to the start of the
// Hebrew calendar to the mean conjunction of Tishri of Hebrew fYear.
class function THebrewDate.HebrewCalendarElapsedDays(y: integer): integer;
var MonthsElapsed,
    PartsElapsed,
    HoursElapsed,
    ConjunctionDay,
    ConjunctionParts,
    AlternativeDay : integer;
begin
  MonthsElapsed :=
    (235 * ((y - 1) div 19))           // Months in complete cycles so far.
    + (12 * ((y - 1) mod 19))          // Regular months in this cycle.
    + (7 * ((y - 1) mod 19) + 1) div 19; // Leap months this cycle
  PartsElapsed := 204 + (793 * (MonthsElapsed mod 1080));
  HoursElapsed :=
    5 + (12 * MonthsElapsed) + (793 * (MonthsElapsed  div 1080))
    + (PartsElapsed div 1080);
  ConjunctionDay := 1 + (29 * MonthsElapsed) + (HoursElapsed div 24);
  ConjunctionParts := (1080 * (HoursElapsed mod 24)) + (PartsElapsed mod 1080);
  if ((ConjunctionParts >= 19440)         // If new moon is at or after midday,
      OR (((ConjunctionDay mod 7) = 2)    // ...or is on a Tuesday...
          and (ConjunctionParts >= 9924)  // at 9 hours, 204 parts or later...
          and not (IsLeapYear(y)))        // ...of a common year,
      OR (((ConjunctionDay mod 7) = 1)    // ...or is on a Monday at...
          and (ConjunctionParts >= 16789) // 15 hours, 589 parts or later...
          and (IsLeapYear(y - 1)))) then  // at the end of a leap year
    // Then postpone Rosh HaShanah one Day
    AlternativeDay := ConjunctionDay + 1
  else
    AlternativeDay := ConjunctionDay;
  if (((AlternativeDay mod 7) = 0)          // If Rosh HaShanah would occur on Sunday,
      OR ((AlternativeDay mod 7) = 3)       // or Wednesday,
      OR ((AlternativeDay mod 7) = 5)) then // or Friday
    // Then postpone it one (more) Day
    result := (1 + AlternativeDay)
  else
    result := AlternativeDay;
end;

// True if y is an Hebrew leap year
class function THebrewDate.IsLeapYear(y: integer): boolean;
begin
  result := ((((7 * y) + 1) mod  19) < 7);
end;

// Last fDay of Month in Hebrew Year.
class function THebrewDate.LastDayOfMonth(m, y: integer): integer;
begin
  if ((m = 2)
      OR (m = 4)
      OR (m = 6)
      OR ((m = 8) and not (LongHeshvan(y)))
      OR ((m = 9) and ShortKislev(y))
      OR (m = 10)
      OR ((m = 12) and not (IsLeapYear(y)))
      OR (m = 13)) then
    result := 29
  else
    result:= 30;

end;

// Last Month of Hebrew Year y.
class function THebrewDate.LastMonthOfYear(y: integer): integer;
begin
  if (IsLeapYear(y)) then
    result := 13
  else
    result := 12;
end;

// True if Heshvan is long in Hebrew fYear.
class function THebrewDate.LongHeshvan(y: integer): boolean;
begin
  result := ((DaysInYear(y) mod 10) = 5);
end;

class function THebrewDate.MonthName(m,y: integer): string;
begin
result:='Invalid Month';
if not IsLeapYear(y) then
   case m of
   1 : result := 'Nissan';
   2 : result := 'Iyar';
   3 : result := 'Sivan';
   4 : result := 'Tammuz';
   5 : result := 'Av';
   6 : result := 'Elul';
   7 : result := 'Tishrei';
   8 : result := 'Heshvan';
   9 : result := 'Kislev';
   10: result := 'Teves';
   11: result := 'Shevat';
   12: result := 'Adar';
   end
else
   case m of
   1 : result := 'Nissan';
   2 : result := 'Iyar';
   3 : result := 'Sivan';
   4 : result := 'Tammuz';
   5 : result := 'Av';
   6 : result := 'Elul';
   7 : result := 'Tishrei';
   8 : result := 'Heshvan';
   9 : result := 'Kislev';
   10: result := 'Teves';
   11: result := 'Shevat';
   12: result := 'Adar I';
   13: result := 'Adar II';
   end
end;

// True if Kislev is short in Hebrew fYear.
class function THebrewDate.ShortKislev(y: integer): boolean;
begin
  result := ((DaysInYear(y) mod 10) = 3);
end;

{ TBaseDate }

function TBaseDate.CalDateAsAbsoluteDate: integer;
begin
result := self.cf_CalDateAsAbsDate(Month,Day,Year);
end;

end.

SMTP Mail and Indy (again)

2008-04-08T16:41:00.001-07:00

Having spent a lot of time recently working on a ping scanner using Indy, I noticed that there's a lot of questions still on using SMTP with Indy. Let me first say that I am not an Indy expert. I get along with it successfully but find I still have to research things frequently.

With the disclaimer out of the way, we can get to the offering. A while back I wrote a handy little unit to simply send a mail message with or without attachments and with or without providing authentication. It even has support for OpenSSL. I use this unit in a number of applications and have seen it successfully send hundreds of e-mails without issue. I recently added support for threaded message sending to take advantage of the multi-core system I'm now running on.

This code has had a few additions (like the logger) that I've gleaned over time from various newsgroup postings, but I didn't record the authors so let me credit the anonymous Internet authors who support Indy. It's really amazing how helpful Google can be when you're stuck. At any rate, here's the code for the mailer in the hopes that it will help someone else who is stuck dealing with SMTP.


unit MailIt;

{requires indy10 - the LATER VERSIONS}

interface

uses Classes, SysUtils, IdSMTP, IdLogFile;

type
  //this class encapsulates the "where to" and "what" portions of the message
  //basically, it sends the BODY, including the files in the FILES list, using the SUBJECT to every person on the SendTo, etc. list
  //SendTo, CCSendTo, and BccSendTo all take comma separated addresses (i.e, SendTo:='first@me.com,second@me.com';)
  TDestinationPart = class
                     SendTo    : string; //list of addresses
                     CCSendTo  : string;
                     BccSendTo : string;
                     ReturnAddy: string; //if this is blank, there will be no return receipt

                     Subject   : string;
                     Body      : string;
                     Files     : TStringList;

                     constructor Create(aSndTo, aCCSndTo, aBCCSndTo, aRtrnRcptAddy, aSubject, aBody:string; aFiles:TStringList);
                     destructor Free;
                     end;

  //this class encapsulates the "how do i get it there" and "who from" portion of the message
  //since this doesn't frequently change, you can use CreateFromFile and SaveToFile to create a template
  //   and save the typing every time you set it up
  TOriginPart = class
                AuthType : TIdSMTPAuthenticationType;
                FromAddy : string;
                UserName : string;
                Password : string;
                Server   : string;
                Port     : integer;
                Debug    : boolean;
                constructor Create(aAuthType:TIdSMTPAuthenticationType; aFromAddy, aUserName, aPassword, aServer: string; aPort:integer);
                constructor CreateFromFile(fn:string);
                procedure SaveToFile(fn:string);
                end;

  //this is the entire message
  //you provide a DestinationPart and OriginPart and send the file using SendNow.
  //   Check SentOk afterwards to make sure it went.
  //   Any error message will be in SendMsg
  TMailMessage = class
                 public
                 Destination : TDestinationPart;
                 Origin      : TOriginPart;
                 SentOk      : boolean;
                 SentMsg     : string;
                 logFile     : TIdLogFile;

                 constructor Create(Dest:TDestinationPart; Orig:TOriginPart);
                 procedure SendNow;
                 destructor Free;
                 end;

  //this REPLACES TMailMessage with a Threaded version.
  //You may want to use the FreeOnTerminate flag to create a "Fire and Forget" threaded solution
  //   NOTE: DO NOT GO CRAZY
  //         Pay attention to the documentation of the OS (try this for what I mean http://blogs.msdn.com/oldnewthing/archive/2005/07/29/444912.aspx)
  //         If you've got a really great quad-cpu quad-core rig, you may want to use a bigger number
  TThreadMailMessage = class(TThread)                           //you get no error results or other
                       protected                                //information from this class, it just tries to send the message in the
                          MailMsg : TMailMessage;               //background.make sure you use the debug function if you're having problems
                          procedure Execute; override;
                       public
                          constructor Create(aDestinationPart:TDestinationPart;
                                             aOriginPart:TOriginPart);
                       end;


//NOTE: The following utility entries use CriticalSections. DO NOT HAMMER THEM.

//You can test this to see if all the TThreadedMailMessages have completed
function MailThreadsDone:boolean;
//A count of the number of TThreadedMailMessages are still out there
function ActiveMailThreadCount:integer;

implementation

uses IdGlobal, IdResourceStringsCore, IdGlobalProtocols, IdResourceStrings, IdExplicitTLSClientServerBase,
    IDPOP3, IdMessage, IdEmailAddress, IdWinSock2, IdIOHandler, IdSSLOpenSSL, IdException,
    IdAttachmentFile, IdText, IdSASL_CRAM_MD5, IdUserPassProvider, IniFiles,
    IdIOHandlerStream, Dialogs, windows;

const
  logFileName='mailit.log'; //the default log file name

var
  ActiveMailThreads : integer=0;
  MailThreadCSR     : TRTLCriticalSection;

// ---------------------------------------------------------------------------

type
 // this is a descendant of TidLogFile, it will create a plain text file with
 // information about the transfer session
 TlogFile = class(TidLogFile)
 protected
   procedure LogReceivedData(const AText, AData: string); override;
   procedure LogSentData(const AText, AData: string); override;
   procedure LogStatus(const AText: string); override;
 public
   procedure LogWriteString(const AText: string); override;
   class function buildLogLine(data, prefix: string) : string;
 end;

// this ensures the output of error and debug logs are in the same format, regardless of source
class function TlogFile.buildLogLine(data, prefix: string) : string;
begin

 data := StringReplace(data, EOL, RSLogEOL, [rfReplaceAll]);
 data := StringReplace(data, CR,  RSLogCR,  [rfReplaceAll]);
 data := StringReplace(data, LF,  RSLogLF,  [rfReplaceAll]);

 result := FormatDateTime('yy/mm/dd hh:nn:ss', now) + ' ';
 if (prefix <> '') then
   result := result + prefix + ' ';
 result := result + data + EOL;
end;

// ---------------------------------------------------------------------------

procedure TlogFile.LogReceivedData(const AText, AData: string);
begin
 // ignore AText as it contains the date/time
 LogWriteString(buildLogLine(Adata, '<<'));
end;

// ---------------------------------------------------------------------------

procedure TlogFile.LogSentData(const AText, AData: string);
begin
 // ignore AText as it contains the date/time
 LogWriteString(buildLogLine(Adata, '>>'));
end;

// ---------------------------------------------------------------------------

procedure TlogFile.LogStatus(const AText: string);
begin
 LogWriteString(buildLogLine(AText, '**'));
end;

// ---------------------------------------------------------------------------

procedure TlogFile.LogWriteString(const AText: string);
begin
 // protected --> public
 inherited;
end;
                                                     
// ---------------------------------------------------------------------------



// ---------------------------------------------------------------------------
{TMailMessage}
// ---------------------------------------------------------------------------

constructor TMailMessage.Create(Dest:TDestinationPart; Orig:TOriginPart);
begin
Destination:=Dest;
Origin:=Orig;
//the class expects these, it will create them by default
if Destination=nil then Destination:=TDestinationPart.Create('','','','','','',TStringList.Create);
if Origin=nil      then Origin:=TOriginPart.Create(satDefault,'','','','',0);
end;

// ---------------------------------------------------------------------------
procedure TMailMessage.SendNow;
var MsgSend:TIdMessage;
   SMTP:TIdSMTP;
   ix:integer;
   SASLLogin:TIdSASLCramMD5;
   UserPassProv:TIdUserPassProvider;
   textPart:TIdText;
begin
  logFile := nil;                                                     //init to no logging
  MsgSend:=TIdMessage.Create;                                         //create the message
  with MsgSend do
     begin
        Body.Text:=Destination.Body;
        if POS('<html>',Body.Text)<>0 then
           if (Destination.Files<>nil) AND (Destination.Files.Count>0) then //if you are using attachments with html mail, you have do do it in sections
              begin
              Body.Text:='';                                          //clear the body
              contentType:='multipart/alternative';                   //setup the message parts
              textPart:=TIdText.Create(MsgSend.MessageParts,nil);     // this is for plain text
              textPart.ContentType:='text/plain';                     // set the content type
              textPart.Body.Add('This message is formatted as HTML.');// just tell the user that the message is html - it would be nice to strip the html, but you'd never make it look right. probably should add support for a plain text and html text in the Destination class
              textPart.Body.Add(Destination.Body);                    // add the plain text part which is really html formatted
              textPart:=TIdText.Create(MsgSend.MessageParts,nil);     // this is for html text (which is basically just plain text with special rules)
              textPart.ContentType:='text/html';                      // set the content type - this is the magic that kicks in the special rules
              textPart.Body.Add(Destination.Body);                    // add the html formatted text
              end
           else
              contentType:='text/html';                               //you can just send it as an html message
        From.Text := Origin.FromAddy;                                 //setup the from address
        Recipients.EMailAddresses := Destination.SendTo;              //setup who it's going to
        Subject := Destination.Subject;                               //set the subject
        Priority := mpNormal;                                         //set the priority (note that you could add to Destination to support different priorities)
        CCList.EMailAddresses := Destination.CCSendTo;                //set the CC list
        BccList.EMailAddresses := Destination.BCCSendTo;              //set the BCC list
        ReceiptRecipient.Text := Destination.ReturnAddy;              //set the return receipt address
     end;

  for ix:=0 to Destination.Files.Count-1 do                            //if we got a list of file names, add each file as an attachment
     TIdAttachmentFile.Create(MsgSend.MessageParts, Destination.Files.Strings[ix]);

  try
    SMTP:=TIdSMTP.Create;                                              //create the SMTP object

    try
      TIdSSLContext.Create.Free;                                       //try to create a SSL context (immediately disposes of it) - need OpenSSL to support this, if it fails, it will fall into the exception
      smtp.IOHandler := TIdSSLIOHandlerSocketOpenSSL.Create(smtp);     //if we succeed, then we can create a SSL socket handler and assign it to the IOHandler
      smtp.UseTLS := utUseExplicitTLS;                                 //and we can mark the we can use tls
    except
      smtp.IOHandler := TIdIOHandler.MakeDefaultIOHandler(smtp);       //the SSL failed to create, so make the default IO handler
      smtp.UseTLS := utNoTLSSupport;                                   //mark that we have no TLS support
    end;
    smtp.ManagedIOHandler := True;                                     //either way, we now have a ManagedIOHandler

    if Origin.Debug then                                               //if we marked that we wanted debugging, we need to do some setup
       begin
       logFile := TLogFile.Create(nil);                                //create the log file
       logFile.FileName:=logFileName;                                  //set the default log file name - NOTE: you could get fancy with this, but I normally don't use it
       logFile.Active:=true;                                           //activate the log file (creates the file and intializes everything)
       smtp.IOHandler.Intercept := logFile;                            //set the logFile to the intercept of the IOHandler to capture the events
       smtp.IOHandler.OnStatus  := smtp.OnStatus;                      // - not sure why we do this, copied code -
       end;

    try
      SMTP.AuthType := Origin.AuthType;                                //setup the auth type : satNONE, satDEFAULT, satSASL
      SMTP.UserName := Origin.UserName;                                //setup the user name
      SMTP.Password := Origin.Password;                                //setup the password

      userPassProv:=TIdUserPassProvider.Create;                        //create the user/pass provider - this handles the login functions for SASL
      userPassProv.UserName:=SMTP.UserName;                            // setup user name
      userPassProv.Password:=SMTP.Password;                            // & password

      if SMTP.AuthType=satSASL then                                    //if we are going to use SASL - basically, SASL just encrypts the login, everything else is still clear text
        begin
        SASLLogin:=TIdSASLCramMd5.Create;                              // create the CramMd5 provider - see http://en.wikipedia.org/wiki/CRAM-MD5 - this is the encryptor
        SASLLogin.UserPassProvider:=userPassProv;                      // assign the user pass provider to it
        SMTP.SASLMechanisms.Add.SASL:=SASLLogin;                       // add the SASL login back to the SMTP object
        end;

      {General setup}
      SMTP.Host := Origin.Server;                                      //setup the server name
      SMTP.Port := Origin.Port;                                        //setup the server port

      {now we send the message}
      SMTP.Connect;                                                    //connect to the server - this will automatically use TLS if configured and supported
      if not SMTP.Authenticate then                                    //authenticate with the server - this will automatically use SASL if configured and supported
         begin
         MessageDlg('An error occurred attempting to send your e-mail. The error was : Unable to authenticate.', mtError, [mbOK], 0);
         exit;
         end;
      try
         try
            SMTP.Send(MsgSend);                                        //send the message
            SentOk:=true;                                              //indicate success
         finally
            SMTP.Disconnect;                                           //disconnect from the server
         end;
      except on E :Exception do                                        //if we had a failure, say so
         begin
         MessageDlg('An error occurred attempting to send your e-mail. The error was : '+E.Message, mtError, [mbOK], 0);
         SentOk:=false;                                                //set failure
         SentMsg:=E.Message;                                           //and grab the error message
         end;
      end;
    finally
    SMTP.Free;                                                         //free the memory
    end;
  finally
     if logFile<>nil then                                              //if we had a log file, free that
        logFile.Free;
     MsgSend.Free;                                                     //free the message
  end;
end;

// ---------------------------------------------------------------------------
destructor TMailMessage.Free;
begin
Destination.Free;                                                      //free the destination
Origin.Free;                                                           //free the origin
end;

// ---------------------------------------------------------------------------


// ---------------------------------------------------------------------------
{TDestinationPart}
// ---------------------------------------------------------------------------

constructor TDestinationPart.Create(aSndTo, aCCSndTo, aBCCSndTo, aRtrnRcptAddy, aSubject, aBody:string; aFiles:TStringList);
begin
SendTo    := aSndTo;          //send message to, comma separate list for multiple adddesses
CCSendTo  := aCCSndTo;        //CC message to
BccSendTo := aBCCSndTo;       //BCC message to
ReturnAddy:= aRtrnRcptAddy;   //leave blank for not return receipt. set if you want a return receipt (me@mydomain.com)

Subject   := aSubject;        //what's it about
Body      := aBody;           //the text, supports HTML or TEXT
Files     := aFiles;          //a list of files to upload - WATCH THIS - you are giving up control by passing in the list, this list is free'd in the destructor
if Files=nil then
  Files:=TStringList.Create; //we must have a list, create if not provided
end;

// ---------------------------------------------------------------------------
destructor TDestinationPart.Free;
begin
Files.Free;                   //free space claimed in create - watch this, it takes control of the list passed in the create and will dispose of it, can cause A/Vs
end;

// ---------------------------------------------------------------------------

// ---------------------------------------------------------------------------
{TOriginPart}
// ---------------------------------------------------------------------------

constructor TOriginPart.Create(aAuthType:TIdSMTPAuthenticationType; aFromAddy, aUserName, aPassword, aServer: string; aPort:integer);
begin
AuthType := aAuthType;    //options are : 0=satNONE, 1=satDEFAULT, 2=satSASL
FromAddy := aFromAddy;    //me@mydomain.com
UserName := aUserName;    //me
Password := aPassword;    //mypassword
Server   := aServer;      //smtp.mydomain.com
Port     := aPort;        //SMTP connection port (25 is default)
Debug    := false;        //debugging off/on
end;

// ---------------------------------------------------------------------------
constructor TOriginPart.CreateFromFile(fn:string);
var INF:TIniFile;
   i:integer;
begin
INF:=TIniFile.Create(fn);
i        := INF.ReadInteger('SMTP','AuthType',ord(satDefault));
case i of
0 : AuthType := satNONE;
1 : AuthType := satDEFAULT;
2 : AuthType := satSASL;
end;
FromAddy := INF.ReadString ('SMTP', 'FromAddy', ''); //me@mydomain.com
UserName := INF.ReadString ('SMTP', 'UserName', ''); //me
Password := INF.ReadString ('SMTP', 'Password', ''); //mypassword
Server   := INF.ReadString ('SMTP', 'Server', '');   //smtp.mydomain.com
Port     := INF.ReadInteger('SMTP', 'Port', 25);     //default SMTP is 25
Debug    := INF.ReadBool   ('SMTP', 'Debug', false); //we don't we debugging on
INF.Free;
end;

// ---------------------------------------------------------------------------
procedure TOriginPart.SaveToFile(fn:string);
var INF:TIniFile;
begin
INF:=TIniFile.Create(fn);
INF.WriteInteger('SMTP', 'AuthType', ord(AuthType)); //options are : 0=satNONE, 1=satDEFAULT, 2=satSASL
INF.WriteString ('SMTP', 'FromAddy', FromAddy);      //me@mydomain.com
INF.WriteString ('SMTP', 'UserName', UserName);      //me
INF.WriteString ('SMTP', 'Password', Password);      //mypassword
INF.WriteString ('SMTP', 'Server',   Server);        //smtp.mydomain.com
INF.WriteInteger('SMTP', 'Port',     Port);          //SMTP connection port
INF.WriteBool   ('SMTP', 'Debug',    Debug);         //debugging off/on
INF.Free;
end;

// ---------------------------------------------------------------------------


// ---------------------------------------------------------------------------
{ TThreadMailMessage & util functions for it}
// ---------------------------------------------------------------------------

procedure IncMailThreadCount;
begin
  EnterCriticalSection(MailThreadCSR);
  inc(ActiveMailThreads);
  LeaveCriticalSection(MailThreadCSR);
end;

procedure DecMailThreadCount;
begin
  EnterCriticalSection(MailThreadCSR);
  dec(ActiveMailThreads);
  LeaveCriticalSection(MailThreadCSR);
end;

function MailThreadsDone:boolean;
begin
  result:=ActiveMailThreadCount=0;
end;

function ActiveMailThreadCount:integer;
begin
  EnterCriticalSection(MailThreadCSR);
  result:=ActiveMailThreads;
  LeaveCriticalSection(MailThreadCSR);
end;


constructor TThreadMailMessage.Create(aDestinationPart: TDestinationPart;
 aOriginPart: TOriginPart);
begin
inherited Create(false);
IncMailThreadCount;
if aDestinationPart=nil then raise Exception.Create('You must supply a destination');
if aOriginPart=nil      then raise Exception.Create('You must supply an origin');
FreeOnTerminate:=true;

MailMsg:=TMailMessage.Create(aDestinationPart, aOriginPart);
end;

procedure TThreadMailMessage.Execute;
begin
  try
     MailMsg.SendNow;
  finally
     DecMailThreadCount;
  end;
end;

initialization
  InitializeCriticalSection(MailThreadCSR);

finalization
  DeleteCriticalSection(MailThreadCSR);

end.

Holidays

2008-03-31T15:49:00.000-07:00

Anyone else out there ever run into the need for a simple little list of holidays? I do a fair amount of stuff that involves questions like "Are we off on such and such a day?" where you have to figure out, "When is Easter this year anyway?" That's pretty easy if you look it up on a calendar. Now try doing that in code.

At any rate, I wrote a tiny little class to provide some basic holidays out. Help yourself to the code, it's not complicated, just annoying. The biggest trick is finding the information in the first place. This is US-oriented, but I'd be glad to extended it if someone wants some other days in there. It would be helpful if you could provide the holiday information you're interested in though (i.e., My Personal Holiday is the 3rd Tuesday of May or whatever). Oh, one last caveat. If the date doesn't appear (like Inauguration day), the DateTime will be a 0. That's fine for comparisons, but if you just do a dump of the dates, it gives you 1899.


unit holidayDateTime;

interface

uses sysutils;

type
   THolidayClass = class
     public
        YearIs       : word;
        NewYears     : TDateTime;
        MLKJr        : TDateTime;
        Inauguration : TDateTime;
        Washington   : TDateTime;
        ArmedForces  : TDateTime;
        Memorial     : TDateTime;
        FlagDay      : TDateTime;
        Independence : TDateTime;
        Labor        : TDateTime;
        Columbus     : TDateTime;
        Veterans     : TDateTime;
        Thanksgiving : TDateTime;
        Christmas    : TDateTime;
        Secretaries  : TDateTime;
        AprilFools   : TDateTime;
        Earth        : TDateTime;
        Fathers      : TDateTime;
        Groundhog    : TDateTime;
        Halloween    : TDateTime;
        Lincoln      : TDateTime;
        Mothers      : TDateTime;
        StPatty      : TDateTime;
        UnitedNations: TDateTime;
        Valentines   : TDateTime;
        Election     : TDateTime;
        Flag         : TDateTime;
        Easter       : TDateTime;
        constructor Create(aYear:word);
        procedure changeYear(aYear:word);
        function getEaster:TDateTime;
        class procedure SafeMonth(var month, y:word);
        class function getDOW(year, month, dow, weekNum:word):TDateTime;
        class function getLastDOW(year, month, dow:word):TDateTime;
     end;

implementation

class procedure THolidayClass.SafeMonth(var month, y:word);
begin
while month>12 do
   begin
   dec(month,12);
   inc(y);
   end;
end;

class function THolidayClass.getDOW(year, month, dow, weekNum:word):TDateTime;
var i:word;
begin
result:=EncodeDate(year, month, 1);
for i:=0 to 7 do
   if DayOfWeek(result+i)=dow then
      begin
      result:=result+i;
      result:=result+((weeknum-1)*7); //move to the correct week, 1st week is inc of 0
      break;
      end;
end;

class function THolidayClass.getLastdow(year, month, dow:word):TDateTime;
var i:word;
begin
//Year has to be on the stack, SafeMonth could change year
inc(month);
SafeMonth(month,year);
result:=EncodeDate(year,month,1)-1;
for i:=0 to 7 do
   if DayOfWeek(result-i)=dow then
      begin
      result:=result-i;
      break;
      end;
end;

constructor THolidayClass.Create(aYear:word);
begin
ChangeYear(aYear);
end;

function THolidayClass.getEaster:TDateTime;
var a,b,c,d,e,f,g,h,i,k,l,m,p  : integer;
    Easter_Month, Easter_Day   : integer;
begin
   a:=yearis mod 19;
   b:=yearis div 100;
   c:=yearis mod 100;
   d:=b div 4;
   e:=b mod 4;
   f:=(b+8) div 25;
   g:=(b-f+1) div 3;
   h:=(19*a+b-d-g+15) mod 30;
   i:=c div 4;
   k:=c mod 4;
   l:=(32+2*e+2*i-h-k) mod 7;
   m:=(a+11*h+22*l) div 451;
   Easter_Month :=(h+l-7*m+114) div 31; //[3=March, 4=April]
   p:=(h+l-7*m+114) mod 31;
   Easter_Day:=p+1; //(day in Easter Month)
   result:=EncodeDate(YearIs,Easter_Month,Easter_Day);
end;


procedure THolidayClass.ChangeYear(aYear:word);
const
   DayOfWeek_Sunday    = 1;
   DayOfWeek_Monday    = 2;
   DayOfWeek_Tuesday   = 3;
   DayOfWeek_Wednesday = 4;
   DayOfWeek_Thursday  = 5;
   DayOfWeek_Friday    = 6;
   DayOfWeek_Saturday  = 7;

begin
YearIs       := aYear;
Easter       := getEaster;
NewYears     := EncodeDate(YearIs,1,1);
MLKJr        := getdow(YearIs, 1, DayOfWeek_Monday, 3);
if (YearIs - 1937) mod 4 = 0 then
   Inauguration := EncodeDate(YearIs, 1, 20)
else
   Inauguration := 0;
Election     := 0;
Flag         := EncodeDate(YearIs,6,14);
Washington   := getdow(YearIs, 2, DayOfWeek_Monday, 3);
ArmedForces  := getdow(YearIs, 5, DayOfWeek_Saturday, 3);
Memorial     := getLastdow(YearIs, 5, DayOfWeek_Monday);
FlagDay      := EncodeDate(YearIs,6,14);
Independence := EncodeDate(YearIs,7,4);
Labor        := getdow(YearIs, 9, DayOfWeek_Monday,1);
Columbus     := getdow(YearIs, 10, DayOfWeek_Monday,2);
Veterans     := EncodeDate(YearIs,11,11);
Thanksgiving := getdow(YearIs, 11, DayOfWeek_Thursday,4);
Christmas    := EncodeDate(YearIs,12,25);
Secretaries  := getLastdow(YearIs, 5, DayOfWeek_Saturday) - 3; //wednesday before last saturday in april
AprilFools   := EncodeDate(YearIs,4,1);

Earth        := EncodeDate(YearIs,4,22);
Fathers      := getdow(YearIs, 6, DayOfWeek_Sunday,3);
Groundhog    := EncodeDate(YearIs,2,2);
Halloween    := EncodeDate(YearIs,10,31);
Lincoln      := EncodeDate(YearIs,2,12);
Mothers      := getdow(YearIs, 5, DayOfWeek_Sunday,2);
StPatty      := EncodeDate(YearIs,3,17);
UnitedNations:= EncodeDate(YearIs,10,24);
Valentines   := EncodeDate(YearIs,2,14);
end;

end.

Delphi Case with Strings

2008-03-28T12:59:00.000-07:00

Zarko Gajic posted about this topic on his Delphi Tips recently showing how one could use a case statement with strings. His solution basically passes in an array on the stack and then iterates through it providing the index number of the matching string. I don't often want to do this, but the idea comes up occassionally enough that I thought I'd play with it a little.

The first thing that struck me with this is that passing things on the stack is bound to be slow. Any time you can avoid memory allocation in your routines, do it. The way Zarko wrote his StringToCaseSelect routine created a COPY of the information on the stack. In my testing, just changing the CaseList: array of string to CaseList:const array of string improved the performance of the code by almost 30% for his example. Mind, I'm not using hyper-precise counters; however, it definitely makes a difference.

Secondly, I was curious how the performance changed if I used the old stand-by: If then else. Using the same testing routine (and imprecise timers), If Then Else is 368% percent faster than the original, non-const version and 283% faster than the const version. That's a pretty hefty speed penalty to pay just for using a case with strings.


procedure TForm1.standardIF;
const strAb = 'About';
      strBl = 'Borland';
      strDl = 'Delphi';
var s:string;
begin
     if s=strAB then begin end
else if s=strBl then begin end
else if s=strDl then begin end
end;

Finally, I started to wonder if there wasn't a faster way to do this. It is likely the iteration and memory operations that are consuming so much time in his example. Trying to reduce this lead me back to everyone's favorite string solution: the hash. CodeGear has a very simple hash routine they use in inifiles (TStringHash.HashOf). Taking that code, I tried using it as a simple


case HashOf('Delphi') of
HashOf('About')   : begin end;
HashOf('Borland') : begin end;
HashOf('Delphi')  : begin end;
end;

but forgot that the items must be constants. Since Delphi doesn't have a precompiler, that leaves me with the ugly task of generating the hash values for each string and then using those constants in my case. That provides me with the following code:


procedure TForm1.delphiHash;
const cnAb = 24272; //HashOf('About');
      cnBl = 389836; //HashOf('Borland');
      cnDl = 92361; //HashOf('Delphi');
begin
//from delphi hash inifiles
case HashOf('Delphi') of
cnAb : begin end;
cnBl : begin end;
cnDl : begin end;
end;
end;

That's quite a bit of work just to use a case with a string, but the performance actually does mean it might be worthwhile. In my tests, I found the case hash style ran 204% faster than the if then else, 577% faster than the const version of StringCaseSelect and 750% faster than the original StringCaseSelect.

Even though all of that is true, I have to admit that the performance difference on a modern CPU wasn't visible until I ran up to the 10,000,000 iterations or more. If you have a rarely-used routine, you could use which ever method suits you. If you have one that gets hammered a lot, the If Then Else style makes the most sense. I cannot really see the point where the payback for doing all the work of the precompiler makes a lot of sense to using the case hash style.

Of course, in my own development I rarely run into a case string need so perhaps you have a need that does make it worth while. If so, keep in mind that a hash does not guarantee uniqueness. It's unlikely you'll hit a duplicate, but not impossible. The less random the data, the more unlikely duplicates become.

Another Vista oddity

2008-03-27T16:23:00.000-07:00

I've assumed, incorrectly apparently, that the Vista "Run as Administrator" function is akin to the sudo or su function in *nix systems. For those not familiar with sudo, it basically provides a one-time elevation of privileges to execute a special command or a command requiring additional user rights. Su basically creates a new security context using the super user and switches into that context (i.e., in a shell, typing su and then the super user password gives you the "super user shell").

Vista does not appear to hold this idea, or at the very least, not entirely. The thing that started me down this path is my use of the subst command. For those who haven't read some of my other articles, I use subst to create a virtual environment letting me work on multiple projects without having to pull the source code down from svn (i.e., c:\source\code1, c:\source\code2, etc. get swaped out to be j:\ for which ever path I'm working with at the time). At any rate, I had my normal command prompt open, substed in the paths I wanted and fired up Delphi 2007. Everything worked great except D2007 locked up everytime I hit a particular bug. Thinking it might be a privledge error causing Delphi to hang, I started Delphi using "Run As Administrator".

Suddenly, nothing would compile. I kept getting error messages saying that the files were missing or locked. Thinking I'd somehow gotten a lock on my files, I rebooted, repeated the process and got the same results. I checked the subst in the command prompt and everything was fine. As a double-check, I then opened a command prompt using "Run As Administrator", typed in subst and voila, no substed drives. Subst in the drives and suddenly everything works as expected.

Apparently, the "Run As Administrator" function has a lot more in common with Terminal Services than it does an elevated privledge in the same session. If you're having problems, you may want to make sure the settings apply equally to the logged in user AND to the "Run As" user.

EDIT:

In another discovery, the network doesn't remember connections either. I have no idea why I thought it would, but it definitely doesn't. If you perform a net use z: \\somedrive\someshare in a normal session, it won't show up in your Run As sessions (and vice versa).

Vista UAC Manifest

2008-03-04T12:25:00.000-08:00

Here's a couple of interesting little tidbits for those using Delphi 2007 and trying to create manifest files to require your program to run as Administrator. For those who don't know what this means, we're talking about running a program and receiving the wonderful little pop-up "A program is trying to take over your life"... well, something like that anyway. You will need your program to receive elevated privileges if you want to do certain things. In my particular instance, I wanted to write to the HKEY_CLASSES_ROOT registry entries. This isn't supposed to be a full reference to what a manifest is, how it works, etc., etc. Microsoft has documented the UAC in some depth and I recommend using their documentation if you want to really understand your options.

First, the manifest structure is basically just an XML document that tells Vista how it should handle the program. For elevated privileges, it should look like:


<assembly xmlns="urn:schemas-microsoft-com:asm.v1" manifestversion="1.0">
<assemblyidentity version="1.0.0.0" processorarchitecture="X86" name="Vista UAC Compat.Application" type="win32">
<description>WindowsVistaReadiness Application</description>
<trustinfo xmlns="urn:schemas-microsoft-com:asm.v3">
<security>
<requestedprivileges>
<requestedexecutionlevel level="requireAdministrator">
</requestedexecutionlevel>
</requestedprivileges>
</security>
</trustinfo>

(thanks to Henrik Bruhn who helpfully posted the above manifest on borland.public.delphi.non-technical)

Second, D2007 automatically includes a manifest file. If you manually create a manifest file, you wind up with two manifests attached to your program. This will actually create a DCC32 error when you go to compile (duplicate resource). In order to fix this, you have to disable run-time themes under Project/Options/Application. Of course, you then become responsible for inserting the appropriate ComCtrl32 language into the manifest to "upgrade" to the themed ComCtrl32 library that became available in Windows XP. Don't ask me why CodeGear didn't give a little check box to enable this feature. It may have something to do with my fifth point below. At any rate, the updated manifest file (merged with what D2007 automatically includes) looks like:


<assembly xmlns="urn:schemas-microsoft-com:asm.v1" manifestversion="1.0">
<assemblyidentity type="win32" name="CodeGear RAD Studio" version="11.0.2804.9245" processorarchitecture="*">
<dependency>
<dependentassembly>
<assemblyidentity type="win32" name="Microsoft.Windows.Common-Controls" version="6.0.0.0" publickeytoken="6595b64144ccf1df" language="*" processorarchitecture="*">
</assemblyidentity>
</dependentassembly>
<trustinfo xmlns="urn:schemas-microsoft-com:asm.v3">
<security>
<requestedprivileges>
<requestedexecutionlevel level="requireAdministrator" uiaccess="false">
</requestedexecutionlevel>
</requestedprivileges>
</security>
</trustinfo>

Note that the information in the assemblyIdentity can be changed to match your information instead of just blindly copying D2007's information.

Third, you have to include the manifest in your dpr. Make sure you've saved the XML from above as some known file name (vista.manifest in my case). To do this, you will need to create an RC and RES files. D2007 makes this fairly straight-forward. You don't even have to fool with brcc32 anymore. Just create a blank text file and insert


1 24 vista.manifest

in the file. Save that as vistaprog.rc (do not name it the same as either your project or any file in your project, this will cause a resource file conflict.) In your dpr, underneath the


{$R *.res}

add a new line


{$R 'vistaprog.res' 'vistaprog.rc'}

Now, when you do your build, D2007 will automatically compile the RC file, include the vista.manifest and generate a RES file which will be linked into the EXE.

Fourth, you cannot run your program from a substituted (using the subst command) drive. This is probably a minor annoyance for most since subst is used by command line junkies more than anyone else. In my specific case, I have a set of code paths representing versions (i.e., c:\source\v1, c:\source\v2, c:\source\v3) and I use subst to create a working environment that the IDE works with but is totally transparent to which version I'm in. For instance, subst j: c:\source\v? where ? represents the source code version I want to work. This provides a drive letter (j:) that is anchored to that directory. If you do try to run the program from a substed drive, you will see that you get a message "The specified path does not exist. Check the path and try again." If you ran it from the console, you receive the same error message AND a command line message saying "The system cannot find the file xxxx." It works fine if you just move the exe to a normal drive.

There may be a fifth issue, I haven't quite decided on this one yet. I am currently unable to run an elevated program in D2007. I suspect this is because Vista starts the exe, sees the manifest, terminates the exe and restarts it under the new security level. Obviously, this is not going to make the Delphi debugger very happy. I'm still researching this particular aspect of it. If this is truly what is happening, it should make debugging elevated programs a lot of fun.

If you are interested in some other articles on the manifest system, look here, the original thread from Henrick on the newsgroup, here for Steve Trefethen talking about the automatic inclusion of manifest info in D2007, here on the XP manifest, here for a discussion on Windows XP manifest and ComCtrl32, and here for a detailed discussion on UAC. The last link I provided offers a COM factory mechanism for elevating your program as needed instead of the whole executable all the time. For my particular purposes, I just needed global elevation, but the COM factory is a much more elegant solution if you only occasionally need access.

Disclaimer: Since I didn't see a step-by-step guide on how to do this anyway, I threw this together as I figured out what was going on. If you have any questions, ask away and I'll try to help.

Introduction

2008-02-22T10:35:00.000-08:00

This is my first attempt to create a blog of my thoughts on coding and technology. I've been developing software for many years now and have an interest in many of the current (and not so current) languages and technologies. I primarily develop with CodeGear's Delphi and have used its predecessors all the way back to Turbo Pascal 3.0.

I have specialized in health care technology so some of my posts may be about that industry in specific, while others will discuss various techniques I find either interesting or useful.

Hopefully, the reader will find the information either relevant or useful. With a touch of luck, perhaps even both.