Welcome, Guest
Username: Password: Remember me
Visual Objects

Please use this forum to post questions about Visual Objects and Vulcan.NET
  • Page:
  • 1

TOPIC:

VO2740: conversion utf8 to Ansii: MemFree error 23 Nov 2021 13:04 #20496

  • kitz


  • Posts: 61
  • Hello,
    I try to convert strings which may contain UTF-8 characters to Windows 1252 (German) encoding with this function:
    FUNCTION ConvertFromCodePageToCodePage(cString AS STRING, dwFrom AS DWORD, dwTo AS DWORD) AS STRING
    	LOCAL pUStr                  AS PTR
    	LOCAL pBuffer, pBuf1, pBuf2        AS PTR
    	LOCAL nLen, nLenRes, nULen, nULenRes     AS LONG
    	// Convert VO string from dynamic memory to fixed memory
    	nLen     := LONG(SLen(cString))
    	pBuffer  := MemAlloc(DWORD(nLen))
    	pBuf1 := pBuffer
    	IF pBuffer == NULL_PTR
    		BREAK "MemAlloc error"
    	ENDIF
    	MemCopyString(pBuffer,cString,DWORD(nLen))
    	
    	// Determine length of Unicode string
    	// And allocate enough space to hold it
    	nULen    := MultiByteToWideChar(dwFrom,0,pBuffer,nLen,NULL_PTR, 0)
    	IF nULen == 0
    		BREAK "MultiByteToWideChar GetLen error"
    	ENDIF
    	IF pBuffer <> pBuf1
    		BREAK "pBuffer modified"
    	ENDIF
    	pUStr    := SysAllocStringLen(NULL_PTR,DWORD(nULen))
    	IF pUStr == NULL_PTR
    		BREAK "SysAllocStringLen error"
    	ENDIF
    	// Convert Fixed memory Ansi string to Fixed memory Unicode string
    	nULenRes := MultiByteToWideChar(dwFrom,0,pBuffer,nLen,pUStr,nULen)
    	IF nULen <> nULenRes
    		BREAK "MultiByteToWideChar Copy Len<>"
    	ENDIF
    	// Now determine size needed for ANSI string
    	nLen :=      WideCharToMultiByte(dwTo,0,pUStr,nULen,NULL_PTR,0, NULL,NULL)
    	
    	// Allocate Fixed memory buffer to hold the UTF8 string
    	pBuffer  := MemRealloc(pBuffer, DWORD(nLen+1))
    	IF pBuffer == NULL_PTR
    		BREAK "MemRealloc error"
    	ENDIF
    	pBuf2 := pBuffer
    	// Convert Unicode to Ansi
    	nLenRes  := WideCharToMultiByte(dwTo,0,pUStr,nULen,pBuffer,nLen ,NULL,NULL)
    	IF nLen <> nLenRes
    		BREAK "WideCharToMultiByte Len<>"
    	ENDIF
    	
    	// Convert fixed memory buffer to dynamic memory string
    	cString  := Mem2String(pBuffer,DWORD(nLen))
    	IF pBuffer <> pBuf2
    		BREAK "pBuffer 2 modified"
    	ENDIF
    	// Release fixed memory buffer
    	IF MemFree(pBuffer) <> 0
    		BREAK "MemFree error: " + cString     // <------ here it breaks
    	ENDIF
    	// Release the Unicode String
    	SysFreeString(pUStr)
    	
       RETURN cString
    The MemFree() function near the end doesn't return 0 so according to the vo27 doc it fails!
    Without all the test IF's it failed after some calls with an 5333 error. It works for some calls and fails with others. The string which creates the error is about 400 characters long and contains a handful of UTF-8 characters.
    I absolutely don't understand what's going on. pBuffer is saved and compared before freeing, so it's the same.
    Debugging works regarding breakpoints, but I am not able to look at any variable, neither by hovering nor by ctrl-x. Mostly it shows empty, sometimes a result from a previous assignment to some other variable in the code, so unusable.
    VO2740, Win10 Enterprise 20H2, Windows Information Protection enabled (Company policy)
    Any ideas?
    BR Kurt

    Please Log in or Create an account to join the conversation.

    VO2740: conversion utf8 to Ansii: MemFree error 23 Nov 2021 18:07 #20506

    • robert
    • robert's Avatar


  • Posts: 2475
  • Kurt,
    I am not sure why you are checking pBuffer against pBuf1 and pBuf2. There is no way that the call to WideCharToMultiByte() can change the pointer which is a local variable.
    If MemFree() fails then this is usually an indication that some process has written outside of the boundaries of the allocated memory.
    VO allocates a small extra buffer before and after the memory block that you allocate with MemAlloc().
    MemFree() inspects these extra buffers to see if the are (accidentally) overwritten.
    I suspect that one of the parameters to WideCharToMultiByte() is incorrect and that this function therefore overwrites the extra block of memory at the end of the pbuffer block.

    Robert

    Edit:
    PS Maybe we can create a (new) separate method in the SqlSelect class in which the oStmt is allocated. That would allow you to override that method in your subclass and return an instance of a subclass of SqlStatement.
    If you like this idea, then please create an issue in our GitHub repo, so we will not forget it.
    XSharp Development Team
    The Netherlands

    Please Log in or Create an account to join the conversation.

    Last edit: by robert.

    VO2740: conversion utf8 to Ansii: MemFree error 23 Nov 2021 19:40 #20509

    • kitz


  • Posts: 61
  • Robert,
    thanks for the explanation. I compared everything to be sure it was not overwritten (I got 5333 errors sometimes) as I can't really debug and watch.
    I now avoided this situation by creating a large global byte buffer with MemAlloc() in the start() routine as well as a large global character buffer with SysAllocStringLen() the same way. Then I check the sizes against the buffer sizes and reuse the global buffers because this conversion is called 12 times per record with 100's of records. This seems to work fine.
    If I have time I will check the buffer contents to see what happens.
    I assume, the EDIT part belongs to my other question regarding SQL Classes ;-)
    BR Kurt

    Please Log in or Create an account to join the conversation.

    • Page:
    • 1