Error handling into a table

Developers need to call the uspErrorHandling stored procedure in their code with the optional parameter list.

ALTER PROCEDURE [dbo].[uspErrorLogTest] (@date as datetime)




AS
  --sample exec [uspErrorLogTest] '11/02/2018'
  
BEGIN TRY

declare @paramList varchar(4000) = '@date=' + cast( @date as varchar)

-- SELECT 1/0

select cast(@date as int) /0

END TRY

BEGIN CATCH

begin

EXEC dbo.uspErrorHandling @paramList




 end

END CATCH


if object_id('uspErrorHandling','P') is not null

drop procedure uspErrorHandling




GO
  
CREATE procedure dbo.uspErrorHandling (@ErrorProcParameterList varchar(4000) = null)

AS




-- Declaration statements
  DECLARE @ErrorNumber int

DECLARE @ErrorMessage varchar(4000)

DECLARE @ErrorSeverity int

DECLARE @ErrorState int

DECLARE @ErrorProcedure varchar(200)

DECLARE @ErrorLine int

DECLARE @UserName varchar(200)

DECLARE @HostName varchar(200)

DECLARE @TransactionDateTime datetime




-- Initialize variables
  SELECT @ErrorNumber = isnull(error_number(),0),

@ErrorMessage = isnull(error_message(),'NULL Message'),

@ErrorSeverity = isnull(error_severity(),0),

@ErrorState = isnull(error_state(),1),

@ErrorLine = isnull(error_line(), 0),

@ErrorProcedure = isnull(error_procedure(),''),

@UserName = SUSER_SNAME(),

@HostName = HOST_NAME(),

@TransactionDateTime = GETDATE();

select @ErrorMessage




-- Insert into the dbo.ErrorLog table
  INSERT INTO dbo.ErrorLog (ErrorNumber, ErrorMessage, ErrorSeverity, ErrorState, ErrorLine,

ErrorProcedure, ErrorParameterList,UserName, HostName,TransactionDateTime)

SELECT @ErrorNumber, @ErrorMessage, @ErrorSeverity, @ErrorState, @ErrorLine,

@ErrorProcedure, @ErrorProcParameterList,@UserName, @HostName, @TransactionDateTime


GO


--Table where we log the errors

CREATE TABLE [dbo].[ErrorHandling](

[ErrorHandlingID] [int] IDENTITY(1,1) NOT NULL,

[Error_Number] [int] NOT NULL,

[Error_Message] [varchar](4000) NULL,

[Error_Severity] [smallint] NOT NULL,

[Error_State] [smallint] NOT NULL,

[Error_Procedure] [varchar](200) NOT NULL,

[Error_Line] [int] NOT NULL,

[UserName] [varchar](128) NULL,

[HostName] [varchar](128) NULL,

[TransactionDateTime] [datetime] NOT NULL,

[CreatedDateTime] [datetime] NOT NULL,

PRIMARY KEY CLUSTERED




(
   [ErrorHandlingID] ASC

)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]

) ON [PRIMARY]




GO   

How to read mixed data type from excel file in SSIS

To fix the issue where SSIS was reading mixed data  type wrong from an EXCEL file I had to change the registry information
I had a file where the source file had account numbers where some rows had value as integer and some had alpha numeric character. When the account number had first few rows as integer the remaining rows with the alphanumeric account number value was not getting read and the value was getting imported as NULL. I followed the following steps to fix this issue in EXCEL 2012 .

1. In Windows environments, select Start ► Run and type REGEDIT to display the Registry Editor.
2. In the registry tree, select HKEY_LOCAL_MACHINE ► Software ► Wow6432Node ► Microsoft ► Office ► 14.0 ► Access Connectivity Engine ► Engines.
3. Double-click the Excel node.
4. In the right panel, double-click the TypeGuessRows entry.
5. Change the value data from 8 to 0.
6. Click OK.
7. Select File ► Exit to exit the Registry Editor window.

This modification instructs Excel to scan all of the rows in the table for their data types. This action might decrease performance on very large tables, but the importation of data will be more accurate. You can also set TypeGuessRows from 0 to 16, inclusive, where 0 scans all of the rows and the other numbers set the number of rows to scan first.

How to add the source file row number to the destination table.

https://www.hansmichiels.com/2015/11/13/how-to-add-a-rownumber-to-a-data-flow-task-in-ssis-2012/#comment-205

The above link has all the explanation. It was very helpful.

Generate XML data

--XML data sample to be genereated

<?xml version="1.0"?>
-<serialization SerializationRoot="SBOInboundBatchCO">

-<SBOInboundBatchCO ClassName="com.siemens.med.hs.sf.thirdpartycollection.inbound.SBOInboundBatchCO">

-<sboInboundBatchIO ClassName="com.siemens.med.hs.sf.thirdpartycollection.inbound.SBOInboundBatchIO">
<creationDate Type="DATE">2017/05/02</creationDate>
<externalReferenceDate Type="DATE">2017/05/02</externalReferenceDate>

-<sboInboundMessageIOs ClassName="com.siemens.med.hs.sf.thirdpartycollection.inbound.SBOInboundMessageIO" IsIndexed="1">

-<element ClassName="com.siemens.med.hs.sf.thirdpartycollection.inbound.SBOInboundMessageIO">
<billingPeriodsequenceNumber>2</billingPeriodsequenceNumber>
<messageCodeBehaviourCode Type="STRING">CACISBOACKGMENT</messageCodeBehaviourCode>
<messageText Type="STRING"/>
<receivableGrpId Type="STRING">615466</receivableGrpId>
<responsiblePartyIdentifier Type="STRING">Y</responsiblePartyIdentifier>
</element>

</sboInboundMessageIOs>
<sboName Type="STRING">CACi BD Collections</sboName>
<totalMessageCount Type="NUMERIC">284</totalMessageCount>
</sboInboundBatchIO>
</SBOInboundBatchCO>
</serialization>

Used the following SQL script to get this output

declare @intTotMsgCount int , @xmlstr xml



　
 set @inttotMsgCount = (Select count(Account_number) from Imp_staging_referrals_claims_FollowUp where imp_fileId = @intfileId

and uploaded = 1 and deleted = 0)



　
 set @xmlstr=



　
 (select 'SBOInboundBatchCO' as '@SerializationROOT',




(
  select 'com.siemens.med.hs.sf.thirdpartycollection.inbound.SBOInboundBatchCO' as '@ClassName',




(
  select 'com.siemens.med.hs.sf.thirdpartycollection.inbound.SBOInboundBatchIO' as '@ClassName',

'DATE' as 'creationDate/@Type',

convert(varchar(10),cast(getdate() as date) ,111) as 'creationDate',

'DATE' as 'externalReferenceDate/@Type',

convert(varchar(10),cast(getdate() as date) ,111) as 'externalReferenceDate',




(
  select 'com.siemens.med.hs.sf.thirdpartycollection.inbound.SBOInboundMessageIO' as '@ClassName', '1' as '@IsIndexed',




(
  select ' com.siemens.med.hs.sf.thirdpartycollection.inbound.SBOInboundMessageIO' as '@ClassName',

'1' as 'billingPeriodsequenceNumber',

'STRING' as 'messageCodeBehaviourCode/@Type',

'CACISBOACKGMENT' as 'messageCodeBehaviourCode',

'STRING' as 'messageText/@Type',

'' as 'messageText',

'STRING' as 'receivableGrpId/@Type',

substring(Account_Number,2,len(Account_Number)-4) as 'receivableGrpId',

'STRING' as 'responsiblePartyIdentifier/@Type',

'Y' as 'responsiblePartyIdentifier'

from Imp_staging_referrals_claims_FollowUp

where imp_fileId = @intFileId

and Facility_Code = @intClientCode



　
 for xml path ('element') , elements,type




)
  for xml path ('sboInboundMessageIOs') , type

)

,'STRING' as 'sboName/@Type',

'CACi BD Collections' as 'sboName',

'NUMERIC' AS 'totalMessageCount/@type',

@intTotMsgCount as 'totalMessageCount'

for xml path('sboInboundBatchIO'), elements,type




)
  
for xml path('SBOInboundBatchCO'), type




)
  for xml path ('serialization') , type




)
  --AS COL_XML
  
select '<?xml version="1.0"?>' + cast(@xmlstr as varchar(max)) as COL_XML

How to export data from database tables to an XML file using SSIS

I had a project where I had to generate a xml file with data from database tables using SSIS.

The components involved were

Data Flow Task
OLE DB Source - which had the SQL command at the stored proc that called the XML data
Flat File Destination  that used a Flat File Connection Manager.

The important thing to remember here was the  FFCM set up .
Here are the screen shots to show I set it up .

The DFT consisted of OLEDB Sour Editor calling a procedure that generated XML data.

Below is the Flat File Connection Manager used. The format was DELIMITED.

 

 

In the Columns section a new column COL_XML is added which is the ONLY column returned by the stored procedure that has the XML data.

 

 

 

 



The final important part was the below where we set up the data type which had to be Unicode text stream

Error: Tranaction log full due to replication

I had this error in one of the databases and I was trying different possible steps to fix it.

I tried to take a transactional log back up - no luck
I tried to change the recovery model to simple - no luck
The database does not have replication but it does have CDC enabled. So I tried to disable the CDC I got another error related to metadata
I tried to change the db owner to sa - no luck still error
Then the following helped me to get rid of the error that was saying replication

EXEC sp_repldone @xactid = NULL, @xact_sgno = NULL, @numtrans = 0, @time = 0, @reset = 1;

I tried to change the db owner to sa after running the above - success
I tried to disable the CDC - success
I expanded the log file - success

This is how I was able to resolve the issue.