Customising the SimpleBatch GetBatchheader Method

Question

Question

Thembelani Mlalazi · May 18, 2018

#Business Service #Object Data Model #Ensemble

I have a csv record mapper that reads in a batch and my batch has a header and trailer that changes all the time but it is of fixed length so I override the Get Batch Header method from the simplebatch class to something like below but my problem is when the batch is read in It still tries to bring in another empty batch like the picture below any reason why that is here is my code.The red code is the changes made. After several trace logs it is being called twice first time with the batch [1] and[5]is the second call how can I stop that from happening

/// Get the Batch Header from an incoming stream. This is only invoked once by the RecordMap
/// batch services when a new stream is passed in. Any extra data must be returned in pLookAhead
/// for use in parsing the first record.
ClassMethod GetBatchHeader(pIOStream As %IO.DeviceStream, pTimeout As %Numeric = -1, Output pBatch As EnsLib.RecordMap.SimpleBatch, ByRef pLookAhead As %String) As %Status
{
 Try {
   Set tStatus = $$$OK
   Set pBatch = ""
   Set tTerm = ..GetHeaderTerm()
   //Set tFullHeader = 63 _ tTerm
   Set tHeaderLen =63+$length(tTerm)
  If tHeaderLen
    {
   Set tFound = 0
   Set tLeadingJunk = ""
   Set pLookAhead = $get(pLookAhead)
   Set tTimeout = pTimeout
   Set tEndTime = $zhorolog + pTimeout
   While ('tFound) && ('pIOStream.AtEnd)
     {
     Set tReadLen = tHeaderLen - $length(pLookAhead)
     If tReadLen > 0
        {
        Set tData = pLookAhead _ pIOStream.Read(tReadLen, .tTimeout, .tStatus)
        $$$TRACE("1")
        If $$$ISERR(tStatus) Quit
        If tTimeout
          {
         Set tStatus = $$$ERROR($$$EnsErrTCPReadTimeoutExpired, pTimeout, tReadLen)
         Quit
          }
          Set pLookAhead = ""
       }
    Else
      {
       $$$TRACE("2")
      Set tData = $extract(pLookAhead, 1, tHeaderLen)
      Set pLookAhead = $extract(pLookAhead, tHeaderLen + 1, *)
      }
      If ($extract(tData,1,3) = "001" )
         {
        Set pBatch = ..%New()
        set pBatch.BatchHeader = tData
        Set tFound = 1
        $$$TRACE("Foundlee"_tFound)
        Quit
        }
        Else
          {
           $$$TRACE("3")
         Set pLookAhead = pLookAhead _ tData
          #; Check if we should start discarding leading data
         If ($length(pLookAhead) >= tHeaderLen)
            {
           If ($length(tLeadingJunk) < 400)
              {
             Set tLeadingJunk = tLeadingJunk _ $extract(pLookAhead,1)
              }
              Set pLookAhead = $extract(pLookAhead,2,*)
             }
     
           }
           If (pTimeout = -1)
              {
             Set tTimeout = -1
              }
    Else {
     Set tCurrTime = $zhorolog
     If (tCurrTime > tEndTime) {
      Set tStatus = $$$ERROR($$$EnsErrTCPReadTimeoutExpired, pTimeout, tReadLen)
      Quit
     }
     Set tTimeout = tEndTime - tCurrTime
    }
   } //while
   If $$$ISERR(tStatus) Quit
   #; Clear the lookahead buffer if we didn't find the batch header
   $$$TRACE("###Status###"_tStatus)
   If (('tFound) && ($length(tLeadingJunk) < 400)&&($$$ISERR(tStatus))){
    Set tLeadingJunk = tLeadingJunk _ $get(pLookAhead)
    Set pLookAhead = ""
    $$$TRACE("###Not Found###"_tFound)
   }
   If (tLeadingJunk '= "") && ('..#IgnoreLeadingData) {
    #; Use JS escaping to handle control characters
    Set tLoggedJunk = $zconvert($extract(tLeadingJunk,1,400),"O","JS") _ $select($length(tLeadingJunk) > 400: "...", 1: "")
    $$$LOGWARNING($$$FormatText($$$Text("Discarding unexpected leading data: '%1'","Ensemble"),tLoggedJunk))
   }
   If (('tFound)&&($$$ISERR(tStatus))) {
    Set pBatch = ""
    $$$TRACE("Not Found")
    Set tStatus = $$$ERROR($$$EnsRecordMapErrBatchHeaderNotFound,$classname($this))
    Quit
   }
  }
  Else {
   Set pBatch = ..%New()
   Set pLookAhead = $get(pLookAhead)
   Quit
  }
 }
 Catch ex {
  Set tStatus = $$$EnsSystemError
 }
 Quit tStatus
}

Discussion (5)1

Log in or sign up to continue

Eduard Lebedyuk May 22, 2018 to Thembelani Mlalazi

You can concatenate statuses with:

set sc = $$$ADDSC(sc1, sc2)

Or append statuses with:

set sc = $$$ADDSC(sc, sc2)

0 0

score 0 · Answer 1 · 2018-05-22T09:31:18-04:00

Thembelani Mlalazi · May 22, 2018

@Eduard Lebedyuk was wondering if you can spot what the problem is thanks in advance

0 0

score 0 · Answer 2 · 2018-05-22T10:48:04-04:00

@Eduard Lebedyuk Looked at the class this classEnsLib.RecordMap.Service.BatchFileService and the EnsLib.RecordMap.Service.BatchStandard class apparently these are fine but for some reason the custom class acts as if it goes into a loop.I have put some traces in my code and tried to capture the status and change it to fail if a certain validation fails but that makes the whole service to log an error after successfully delivering the first message .

here are the changes made thanks for your help

ClassMethod GetBatchHeader(pIOStream As %IO.DeviceStream, pTimeout As %Numeric = -1, Output pBatch As EnsLib.RecordMap.SimpleBatch, ByRef pLookAhead As %String) As %Status
{
    $$$TRACE("Begin")
 Try {
  #dim cpStatus as %Status=$$$OK
    Set tStatus = $$$OK
    Set pBatch = ""
    Set tTerm = ..GetHeaderTerm()
    //Set tFullHeader = 63 _ tTerm
    Set tHeaderLen =63+$length(tTerm)
    set stage=""
    If tHeaderLen
       {
       Set tFound = 0
       $$$TRACE("0")
       Set tLeadingJunk = ""
       Set pLookAhead = $get(pLookAhead)
       Set tTimeout = pTimeout
       Set tEndTime = $zhorolog + pTimeout
      // While ('tFound) && ('pIOStream.AtEnd)
        //  {
          Set tReadLen = tHeaderLen - $length(pLookAhead)
          If tReadLen > 0
             {
             Set tData = pLookAhead _ pIOStream.Read(tReadLen, .tTimeout, .tStatus)
             $$$TRACE("1")
             If $$$ISERR(tStatus) Quit
             If tTimeout
                {
                Set tStatus = $$$ERROR($$$EnsErrTCPReadTimeoutExpired, pTimeout, tReadLen)
                Quit
                }
                  Set pLookAhead = ""
             }
             Else
             {
             $$$TRACE("Else treadlen less than 0-[2]")
             Set tData = $extract(pLookAhead, 1, tHeaderLen)
             Set pLookAhead = $extract(pLookAhead, tHeaderLen + 1, *)
             }
             If ($extract(tData,1,3) = "001" )
                 {
                Set pBatch = ..%New()
                set pBatch.BatchHeader = tData
                Set tFound = 1
                $$$TRACE("Foundlee[3]"_tFound)
                Quit
                 }
                 Else
                 {
                $$$TRACE("else $extract(tData1,3)Failed[3]")
                Set pLookAhead = pLookAhead _ tData
                 #; Check if we should start discarding leading data
                If ($length(pLookAhead) >= tHeaderLen)
                   {
                   If ($length(tLeadingJunk) < 400)
                      {
                     Set tLeadingJunk = tLeadingJunk _ $extract(pLookAhead,1)
                      }
                      Set pLookAhead = $extract(pLookAhead,2,*)
                   }
                   set cpStatus=0
                   set stage="Extractfailed" //check here
                  // Quit
                  //Continue
     
                 }
                 If (pTimeout = -1)
                    {
                    Set tTimeout = -1
                    }
                    Else
                    {
                     $$$TRACE("time out ok [4]")
                    Set tCurrTime = $zhorolog
                    If (tCurrTime > tEndTime)
                       {
                      Set tStatus = $$$ERROR($$$EnsErrTCPReadTimeoutExpired, pTimeout, tReadLen)
                      Quit
                       }
                       Set tTimeout = tEndTime - tCurrTime
                   }
                   If $$$ISERR(tStatus) Quit
         // } //while
         
          $$$TRACE("while end"_tFound)
         
          If $$$ISERR(tStatus) Quit
          #; Clear the lookahead buffer if we didn't find the batch header
          $$$TRACE("###Status###[5]"_tStatus)
          If (('tFound) && ($length(tLeadingJunk) < 400)&&('pLookAhead="")&&($$$ISERR(cpStatus)))
             {
             Set tLeadingJunk = tLeadingJunk _ $get(pLookAhead)
             Set pLookAhead = ""
             $$$TRACE("###Not Found###[6]"_tFound)
             }
            If (tLeadingJunk '= "") && ('..#IgnoreLeadingData)
               {
               #; Use JS escaping to handle control characters
               Set tLoggedJunk = $zconvert($extract(tLeadingJunk,1,400),"O","JS") _ $select($length(tLeadingJunk) > 400: "...", 1: "")
               $$$LOGWARNING($$$FormatText($$$Text("[]Discarding unexpected leading data: '%1'","Ensemble"),tLoggedJunk))
               }
               If (('tFound)&&($$$ISERR(cpStatus)))
                  {
                 Set pBatch = ""
                 $$$TRACE("Not Found[8]")
                 //Set tStatus = $$$ERROR($$$EnsRecordMapErrBatchHeaderNotFound,$classname($this))
                 if (stage="Extractfailed" &&($$$ISERR(cpStatus)))
                          {
                          
                        
                         set tStatus=cpStatus
                        
                         $$$LOGWARNING($classname($this)_":: Empty Batch Discarded")
                        
                          set tStatus=$$$OK
                          }
                          else
                          {
                            // Set tStatus = $$$EnsSystemError
                            Set tStatus = $$$ERROR($$$EnsRecordMapErrBatchHeaderNotFound,$classname($this))
                             }
                
                 Quit
                  }
     }
     Else
     {
   Set pBatch = ..%New()
   Set pLookAhead = $get(pLookAhead)
   Quit
     }
     }
 Catch ex
    {
      $$$TRACE("in catch]")
    Set tStatus = $$$EnsSystemError
      quit
      }
      
 Quit tStatus
}

Since this is made up of three classes how do I control the status to send the call with within the block have a look at where I have newStatus and I would like to quit and pass new status

score 0 · Answer 3 · 2018-05-22T09:44:00-04:00

Try to check who calls GetBatchHeader and what it returns.

At a glance, I think it's called from GetBatchObject method of EnsLib.RecordMap.Service.BatchStandard class:

Set tStatus = $classmethod(..BatchClass, "GetBatchHeader", pIOStream, pTimeout, .pBatch, .pLookAhead)
If $$$ISERR(tStatus) Quit

Check that tStatus is not an error, fix GetBatchHeader it if it's an error.

score 0 · Answer 4 · 2018-05-23T04:12:55-04:00

Thembelani Mlalazi May 23, 2018 to Eduard Lebedyuk

thanks that seems to help

0 0