Commit Graph

95 Commits

Author SHA1 Message Date
Clemens Schwaighofer
ee62bd98ee Add auto type convert for DB\IO
set via db options "db_convert_type" as array with "on", "json", "numeric",
"bytea"

"on" only converts know good types: "bool", "int"

"json" will convert json/jsonb to array
"bytea" will decode escaped bytea to string (note: this might change to resource)

"numeric" will convert to float.
NOTE: if a numeric number is too large a covnersion might drop data.
Use with care.

Convert flags can be chagned with dbSetConvertFlag and dbUnsetConvertFlag

All convert flags are in "DB\Options\Convert" as enum.
2023-06-09 17:01:03 +09:00
Clemens Schwaighofer
e27ea3dc9f DB\IO phpdoc update 2023-06-09 10:24:26 +09:00
Clemens Schwaighofer
ec4bf54d81 DB\IO phpdoc layout update
Add line between params and method description
2023-06-09 10:21:02 +09:00
Clemens Schwaighofer
e2e080c404 Bug fix for DB\IO 2023-06-01 10:58:11 +09:00
Clemens Schwaighofer
0b93f9f146 Bug fixes and minor updates
- Removed echo from Support Debug dumpVar call
- deprecated DB\IO toggle dbDebug and changed set/get to be like normal
  ones where set just sets and doesn't return anything
- Renamed the logJsDebug to loggingLevelIsDebug
  (other levels can be checked with ->getLoggingLevel()->includes(Level::...))

Adjusted tests for all changes
2023-06-01 08:40:55 +09:00
Clemens Schwaighofer
83ba48f598 Remove db_debug flag from DB\IO, is set from Logging level 2023-05-31 18:41:13 +09:00
Clemens Schwaighofer
30bb0e8895 phpunit checks and fixes 2023-05-31 16:14:40 +09:00
Clemens Schwaighofer
75c4c98de8 Convert all classes to strict variable types
All variable declarations in all classes have a strict type set

Exception: constants (will be setable from PHP 8.3 on), resources (no type)

Debug\LoggingLegacy is kept as is, will be deprecated
2023-05-31 15:58:06 +09:00
Clemens Schwaighofer
f72055909b Update in class error/warning/etc calls instead of debug calls
For anything that is not debug use the proper reporting level
2023-05-30 18:30:31 +09:00
Clemens Schwaighofer
b0a8783276 Logging class major change, Debug\Support update
old Debug\Logging is in Debug\LoggingLegacy and Debug\Logging extends
Logging\Logging

Logging\Logging is a new class with most of the functionality except
there is no more print/outout to screen, but we use the default log
levels (RFC5424)
The plan is to be a frontend between the old type class and
Monolog\Monolog

Updated all other classes to use new class interface
2023-05-30 18:12:24 +09:00
Clemens Schwaighofer
e45acc412b Add better error reporting to DB\IO for query with params
On error with query with params the query was sent to the server and
if ther query itself is ok but there is a problem with the parameters
a wrong error message ($1 not found) will be returned

Add pg_last_error reporting to catch this too.

Update both error reporting to return not string and prefix combined
but prefix + error string in array

In error return check that both strings are not equal, so we do not
return the same error string twice.

Also default set dbh variable in the PgSQL class to false so it will
skip last error report if there is no dbh set yet.

Bug fix for db query with params debug output. if there are more than 9
entries the $1 of eg $10 is replaced with $1 entry again. Changed to
'#' instead '$' to avoid this.

Other:
ACL\Login: replace EOM with HTML
config.master: replace list() with []
Add single DB tester where we can test single db calls without adding
more to the general test run
2023-05-18 15:14:29 +09:00
Clemens Schwaighofer
2b079ff836 DB\IO: add missing debug query, clean up not needed code
in dbReturn with params on not matching param the system exited on fail
without printing the query making it hard to find where the error is.
Added debug output in case the params count is not matching.
Same move in the dbExecute call

removed param count check from dbReturnRow/dbReturnArray as this check
is done in the dbExecParams call anyway
2023-04-11 10:58:38 +09:00
Clemens Schwaighofer
37201799b5 DB\IO params debug output fix for dbReturn/dbReturnParams calls
Those two calls did not replace the params with values for debug output
2023-04-10 17:20:53 +09:00
Clemens Schwaighofer
c51ceb926e Bug fix for DB\IO params detection
Param detection found too many params, for example '$1'.
Fixed the regex to only allow params that are no preceeded by '
And must start with space/tab, =, (
2023-04-07 14:34:13 +09:00
Clemens Schwaighofer
b4b33d6873 Bug fix for DB\IO returning detection
it was still coded with old one line and not taking in possible
line breaks in the returning code part
2023-04-03 15:02:39 +09:00
Clemens Schwaighofer
7eace1013e DB\IO switch dbReturn, dbReturnParams to NO_CACHE as default
Cache is never used, so to keep memory default lower, lets switch to
NO_CACHE
2023-03-29 09:55:09 +09:00
Clemens Schwaighofer
11a8c6440b Update DB\IO debug output for parameter queries
if value null set "NULL" else convert to string

Update class basic test with various type tests
2023-03-28 16:46:34 +09:00
Clemens Schwaighofer
742cbc31df DB\IO add dbExec*, dbReturn* Params methods
Instead of prepare/execute, add the proper quary_params calls for all
the dbExec*, dbReturn* calls

Also add field types to cursor info
2023-03-28 15:31:07 +09:00
Clemens Schwaighofer
e3bd2c1c3b Remove more _SESSION calls in classes, test updates
Admin\EditBase now has ACL\Login class as mandatory class parameter
Output\Form\Generate has loginAcl array parameter as mandatory
2023-03-10 15:08:56 +09:00
Clemens Schwaighofer
03fbcaecfb Code Clean up, more testing
Remove unused code and classes.
Clean up code to remove all named constant from them and throw
deprecation alerts if used.
Add basic psalm setup in root folder and remove from www folder
2023-03-09 15:55:57 +09:00
Clemens Schwaighofer
1596654149 Moved minimum php version to 8.1
All PostgreSQL calls are now Connection/Resource object types and not
resource
All methods have parameter type set
2023-02-28 17:36:19 +09:00
Clemens Schwaighofer
16c3653cee phpstan updates for Smarty move to composer package 2023-02-22 06:42:53 +09:00
Clemens Schwaighofer
53eef03387 Fixes in DB\IO query detection regex
- start with allow whitespace in front
- returning allows more white space types between keyword and parameter list
2023-02-02 10:27:07 +09:00
Clemens Schwaighofer
5a81445a28 DB\IO table match regex fix
UPDATE with SET can have spaces with variable length before
2023-01-27 11:31:26 +09:00
Clemens Schwaighofer
4bbbd653cd DB\IO fix for regex query detection
Fix for basic query detection:
Simeple starts with
SELECT/WITH/SHOW
INSERT INTO/UPDATE/DELETE FROM
UPDATE

Above does no complex query detection, just if the string starts with this

Fix form table detection for primary key auto set trial.
2023-01-27 11:12:46 +09:00
Clemens Schwaighofer
4c28e6d0ec Change DB\IO Returning check regex
Better regex to only get last returning in query, with proper parameter
check
2023-01-26 16:37:22 +09:00
Clemens Schwaighofer
66b7e81463 Bug fix in DB\IO for EOM string build queries with returning
on EOM string build queries it was not checked that RETURNING could have
no space in front.

Fixed and updated test calls
2023-01-25 16:47:31 +09:00
Clemens Schwaighofer
5a0b09a916 Add DB\IO get prepare cursor array entries 2023-01-05 17:26:26 +09:00
Clemens Schwaighofer
b1be681afb Bug fix in DB\IO for wrong array check
Did not use empty to check if query called hash entry exists
2022-12-28 11:26:56 +09:00
Clemens Schwaighofer
6e59b63791 DB\IO phpdoc and phpstan fixes 2022-12-14 14:18:33 +09:00
Clemens Schwaighofer
b7c6d4b478 DB\IO add unescape bytea data 2022-12-09 16:33:58 +09:00
Clemens Schwaighofer
4b3fbaa309 Updates based on latest phpstan run 2022-09-02 17:00:22 +09:00
Clemens Schwaighofer
1a6c65df0e Minor test updates, comment typo updates, DB_CONFIG_SET for default
$DB_CONFIG_SET is now default current selcted db config instead of
$DB_CONFIG so to not overwrite the array itself
2022-08-05 12:43:57 +09:00
Clemens Schwaighofer
cde29c0362 Update DB\IO dbReturn and update connected tests
Switch the code point for these below for logic reasons
CLEAR_CACHE 1 => 2 (clear cache AFTER END read)
READ_NEW 2 => 1 (clear cache BEFORE first read)

in dbReturn cursor ext array:
remove firstcall entry because it is not needed
add new:
- cache_flag: $cache method call number
- assoc_flag: the assoc read flag from the method call
- cached: if there is data cached in the cursor ext array this is true
- finished: true if the last read was false
- db_read_finished: if true the db read has fiinished (read_rows =
  num_rows)
- read_finished: if true the current read (cache or db) via pos =
  num_rows is done
- log_pos: sequential number for each call with the same query hash
- log: array with current actions done in the last read

Update DB IO class test with all cursor, cursor ext, read single step,
read in loop, read again, etc tests
2022-06-03 11:03:06 +09:00
Clemens Schwaighofer
92ebdb4b9e Update session class and write session clas tester, update DB\IO Tester
DB\IO Tester now has correct testing for pgVersion string compare. Uses
mocked pgVersion return for this

Session class update with full magic set/get or method set/get for
_SESSION var. Also added full testing for this
2022-05-26 09:27:57 +09:00
Clemens Schwaighofer
41cff5e3c6 Remove old comment from DB\IO class about encoding settings 2022-04-08 19:59:37 +09:00
Clemens Schwaighofer
e463f48ad4 Updates for language auto detect, fixes for PHP 7.4
There was a bug in ther SQL interface class where the folder was just
called "Interface" which is not an allowed Namespace name. Renamed to
SqlInterface

Moved the detect lang/etc function used in Form/Generate,
Template/SmartyExtend and Admin/Backend to Language/GetSettings.

Fixed some test class calls
2022-04-07 10:49:28 +09:00
Clemens Schwaighofer
13fb22385b Fix DB\IO return for unset pk name to be always string
Add Test for SELECT typ query and num rows type with dbReturn
2022-03-22 20:17:30 +09:00
Clemens Schwaighofer
b7f594e683 Fix, config master base style/js, fix DB\IO num rows
DB\IO dbReturn also sets internal num_rows, num_fields, field_names so
the normal dbGet* calls can be used after dbReturn call

JAVASCRIPT/STYLESHEET in config.master is now override able from .env
file. Others will follow
2022-03-22 20:11:13 +09:00
Clemens Schwaighofer
105e0d69a1 Phan and phpstan fixes
Note that ACL\Login FIXME will be fixed later do not change any current
functionality
2022-03-16 09:01:33 +09:00
Clemens Schwaighofer
40edbe271d DB\IO updates for vesion info calls
dbVersion only returns nn.n version without any additional info
added new dbVersionNumeric that follows the nnNNnn type of format, eg
90605 or 130006
Added new dbVersionInfo where all the pg_version returned array elements
can be checked. Has a strip override for 'server' version to no strip
additional info
dbVersionInfoParameters returns an array of all possible parameters that
can be called.
Not that if the whole info block is needed pg_version($db->dbGetDbh) is
recommended for now.

Also dbCompareVersion does not call the dbVersion but calls the
functions __dbVersion
2022-03-14 10:42:39 +09:00
Clemens Schwaighofer
3881a2a83d DB\IO fix primary key return
Fixed table lookup if tables are in quotes (")
If no primary key name is set return empty.
If no primary key ID is returned, return null.
2022-03-08 06:14:41 +09:00
Clemens Schwaighofer
cf49bf72fa DB\IO move db related queries to db functions and add interface class
A new interface class for DB\SQL\* functions

Also moved all PostgreSQL related SHOW/SET code to the DB\SQL\*
functions class.
2022-03-07 13:35:54 +09:00
Clemens Schwaighofer
e3cf55efe5 DB\IO update with schema/encoding error throws, async/prepare fix
Async query calls have some additional error checks.
Same for prepared calls (eg missing statement name is now error)
Prepared call also inits cursor to avoid unset array key errors

schema/encoding settings checks more strict with proper error returns.
schema checks in database if schema exists.
encoding just throws error if encoding setting failes. future plan is to
check if given encoding is actually valid
2022-03-04 20:19:22 +09:00
Clemens Schwaighofer
a0a7389d3b Fixes for phan and phpstan
Note that the (string) for pg_escape_string and pg_escape_literal are
forced for phpstan because it thinks this might return false even
thought both function only return string ever
2022-03-03 06:49:15 +09:00
Clemens Schwaighofer
42b961f35e DB\IO Class updates, other Class fixes
Output\Form\Generate and ACL\Login have DB\IO method call name changes
for pos/num rows methods. Use proper methods and not use the cursor full
return check method

DB\IO:
Switched to new conenction busy check with sockt and timeout loop. So
short blocked and psql error blocked ones are not blocking other calls.

Moved the dbReturn cache read to a separte private method and cleaned up
code for more clear view

Moved all query hash creations to method to simple change hash creatio
if needed. This method can be used for external correct query hash
creation if needed.

Variable name and code block clean up in dbReturn

No data return (dbReturn or dbFetchArray) will change returned data row.
Return as is.

Moved methods around in code to group them together for next stage in
sub class creation (planned)

Renamed dbCursorPos to dbGetCursorPos
and dbCursorNumRows to dbGetCursorNumRows

Work on phpunit tests for DB\IO
2022-03-03 06:48:57 +09:00
Clemens Schwaighofer
9d79dd4af4 Update for DB::IO
DB/SQL/Pgsql:
add __dbConnectionBusySocketWait for testing with loop type and socket
check wait loop for async queries. (TODO)
add parameter query method to supplement current normal query only
function

DB/IO:
Make all class variables private and add needed methods to read them.
Also add method for reset query in case for new calls.
db error/warnings are written to proper history file with timestamp,
source, etc.
Update source (where called) to show the whole method stack.
Move connection OK to method to easy switch to different connection open
for queries check type.
Never use method variable query, instead only use method parameter and
error out on not set.
All core calls reset errors and set error hash id for history.
Bug fix for insert data return array and reset said array every time a
query execution is started and not only if we have some return data to
avoid having wrong (previous query) data for a new query.
Add more error info to all erros where a separate db logging call was
done.:

ACL/Login:
Do not access DB IO cursor variable directly but use method to read
data. Fixes change for hash type
2022-03-01 20:22:59 +09:00
Clemens Schwaighofer
aafca0153f PHP Unit testing work for DB::IO
Also various clean ups for DB::IO
- fix PGSQL array to PHP
- add bool/literal escape to SQL
- fix literal escape to call correct php array
- move functions to correct place
2022-02-28 18:06:59 +09:00
Clemens Schwaighofer
7a902b5681 Update hash class unit tests, work on DB IO unit tests 2022-02-25 18:08:10 +09:00
Clemens Schwaighofer
780fdedcfd Add long hash to Hash class, switch DB IO from md5
Add a new long hash type and uniq id long hash.
Also set the default hash (short) and hash long to a public constant

Switch all DB IO from md5 to long hash type so we can easy update
hashing of queries
2022-02-25 13:44:27 +09:00