How many characters can be stored in a field with a long text data type?

Hi Biswajit,

First of all, that's quite a long string you are passing on.

Do you need the string to be that long and do you need advanced SQL to do so?

Regarding the reason for the limit on 2000 characters, I think it is in line with the good practice of avoiding the use of 2000+ characters in a text field as those are saved as binaries in the database.

Kind Regards,
João

Thanks Joao, Basically I am not saving entire text in a DB  field it's a CSV content I am processing that for a bulk insert operation in a table using Stored Proc. I willl see if I can pass that as Binary and process that in stored proc.

Although I understand your point to call a store procedure, I would think on different strategies:

  • Perhaps use a INSERT... SELECT bulk Advanced SQL inserts if possible;
  • Settle for an approach not so "performant" but run the process in the background (like BPTs or timer) to avoid impacting on the user experience.

Both approaches would be more maintainable and traceable moving forward, which would save time in the future when you or somebody else needs to analyse the code.

Kind Regards,
João

Thanks Again, Yeah, I though of BPT as well but actually in our business case the number of records can be somewhere around 2000 to 4000 records which in a asynchronous process of inserting one at a time will take very long to complete the transaction.

Hi Biswajit,

You can launch LBPTs (faster than BPTs) as this is an automatic test and have them run parallel inserting let's say a batch of 10 each. There's a presentation of a previous NextStep edition explaining their approach and evaluating the performance attained which you can watch here.

In this case,  you could have an array of LBPT's inserting 10 at each time and the 4k records would be inserted very quickly.

Take into account that this is assuming that the order on which the records are processed is not important given the parallelism of the Light BPT.

Kind Regards,
João

Hi Shashankit Thakur,

After doing some search for this topic I found this community discussion and also please read out the João Batista reply. Hope this discussion will be a help for your query. 

Kind Regards,

Benjith Sam

Shashankit Thakur wrote:

Hey!

I was trying to store JSON strings using text data type. It will be helpful if I could know what is the maximum size of text data type in outsystems?

Hi Shashankit Thakur,

It depends on the database which you are using. for example if you are using SQL server 12 the max length is 4000 Character.

Raj wrote:

Shashankit Thakur wrote:

Hey!

I was trying to store JSON strings using text data type. It will be helpful if I could know what is the maximum size of text data type in outsystems?

Hi Shashankit Thakur,

It depends on the database which you are using. for example if you are using SQL server 12 the max length is 4000 Character.

Thanks!

Hi,

For strings bigger than 2000 outsystems uses nvarchar(max). 

The max size for a column of type NVARCHAR(MAX) is 2 GByte of storage.

Since NVARCHAR uses 2 bytes per character, that's approx. 1 billion characters

Regards,

Marcelo

Neither this thread, nor the one linked above, answers the question!

Outsystems may USE varchar(max), but if you set it to 4000, would that still limit it to 4000 characters?

Hi Nathan,

Yes Outsystems will limit it.

Regards,

Marcelo

So if you want the maximum space, what length value do you use... the max value for an int is 2,147,483,647, so can we just whack in 999999999 (999 million etc)

Hi Nathan,

Never tried it before but in theory it should work. If you end up doing the experiment share with us the result.

Regards,

Marcelo

In case anyone's interested, we gave it a spin so we could store the diff snapshot of a large form. It took 15 minutes to publish and there's no abort button so we're currently considering a different solution...

Hi,

We do storage of large JSON, or XML strings by defining the attribute data type as Binary. Then using TextToBinary from the Binary espace to convert the text prior to storing it, and BinaryToText to convert it back to a text after reading it. Now you do not have to worry about size property of the attribute. You could decide to redesign your entity and create a 1 on 1 entity just to store the binary separate from the other attributes of your entity, with the primairy key set to the identifier of your original table. 

Ideally OutSystems should support the JSON data type natively for an entity attribute. There is already an idea for that that you can vote for.

https://www.outsystems.com/ideas/5310/JSON+datatype+in+Outsystems?IsFromAdvancedSearch=True

Regards,

Daniel

Hi All,

In my case as Ferreira said, I have increased text length to 2500 and OutSystems / SQL Server automatically created nvarchar(max) in database.

Sravan

Hi All, 

For a Text input parameter in a Process I'm having the error "String or binary data would be truncated. " at the moment of launching the process. 

This is the fragment of the stacktrace which reflects where is the problem launching the process:

String or binary data would be truncated.
The statement has been terminated.
at OutSystems.HubEdition.RuntimePlatform.Processes.ProcessBase.ThrowSpecificException(String failureMessage)
   at OutSystems.HubEdition.RuntimePlatform.GenericExtendedActions.ProcessLaunch(HeContext heContext, ObjectKey SSKey, ObjectKey espaceSSKey, Int32 parentActivityId, Int32 parentProcessId, List`1 inputs, List`1& nextActIds)

There is an additional constraint for the Text data type length when used as input parameter for a Process? the parameter in question had 3486 characters length in this issue.

Thanks in advance

Hi Roberto,

Yes there is a constraint . the limit is 2000. since processes are asynchronous that value is saved in the database and the field is a nvarchar(2000).

Regards,

Marcelo 

Thank you Marcelo for your quick response, I've already solved the problem by creating a record in DB and passing the identifier to the process.

Regars, 

   Roberto

I would also look into the possibility to store these large texts or binaries outside the database. Although storing these is something a database can do you should really ask yourself the question if that is the best place to store it. Storing it in the database will cost performance, is not cost efficient and will not support a lazy-load experience. Perhaps storing it in a Azure Storage Account or an AWS S3 is a much better solution since that is a lot cheaper and more suitable for storing large amount of data.

How many characters can a long text data type store in mysql?

Each LONGBLOB value is stored using a 4-byte length prefix that indicates the number of bytes in the value. A TEXT column with a maximum length of 4,294,967,295 or 4GB (232 − 1) characters. The effective maximum length is less if the value contains multibyte characters.

Is there no maximum field size for a long text data type?

LONGTEXT is a string data type with a maximum length of 4,294,967,295 characters.