1 /++
2     Database “drivers” abstract database specific implementation details through a universal interface.
3 
4     This approach allows to build higher level abstractions on top that are independant from the underlying database.
5 
6 
7     ## User’s Guide
8 
9     …or well, almost: Different database implementations will still use different SQL dialects for their queries.
10 
11     For safety and simplicity reasons, the interface is built around the concept of so-called $(I Prepared Statements).
12     [DatabaseDriverSpec.prepare|db.prepare] returns a $(I Prepared Statement) compiled from the provided SQL code.
13 
14     In those cases where you only want to modify the schema (DDL), talk to the DB server or similar
15     and you don’t expect to retrieve an actual result, [DatabaseDriverSpec.execute|db.execute] does the job.
16 
17     $(WARNING
18         Don’t ever feed user-provided data into [DatabaseDriverSpec.execute|db.execute()] or you might end up
19         vulnerable to SQL injections.
20 
21         This function is only intended for executing ready-made DCL, DDL or TCL statements.
22     )
23 
24 
25     ### Examples
26 
27     #### Basic usage
28 
29     ---
30     // connection setup (driver-specific!)
31     DatabaseDriver db = new SQLite3();
32 
33     // disconnect on exit
34     // alternative: you could also call .close() manually later instead
35     scope(exit) db.close();
36 
37     // establish a fresh database connection
38     db.connect();
39 
40     // use .connected to check whether there is an active connection
41     if (!db.connected) {
42         writeln("Disconnected :(");
43     }
44     ---
45 
46 
47     #### Accessing data
48 
49     through prepared Statements
50 
51     ---
52     // prepare a new statement
53     Statement stmt = db.prepare("SELECT * FROM mytable WHERE id >= ? AND id < ?");
54 
55     // cleanup on exit
56     scope(exit) stmt.close();
57 
58     // set // dynamic parameters are 0-indexed, first param is zero (not one!)
59     stmt.bind(0, 200);
60     stmt.bind(1, 400);
61 
62     // execute the prepared statement
63     stmt.execute();
64 
65     // is there a result?
66     if (!stmt.empty) {
67         // - yes
68         Row row1 = stmt.front;
69         writeln(row1);
70     }
71     else {
72         // - no
73         writeln("No data found");
74     }
75 
76     // advance to the next row
77     stmt.popFront();
78 
79     const s = (!stmt.empty) ? "a" : "no";
80     writeln("There is ", s, " 2nd row");
81     ---
82 
83     ---
84     // bind'n'execute helper
85     stmt.executeWith(30, 90);
86 
87     // loop over the result (Statement is an InputRange)
88     foreach(Row row; stmt) {
89         const firstColumn = row[0];
90         writeln("Entry found: ", firstColumn);
91     }
92     ---
93 
94 
95     #### Transactions
96 
97     ---
98     // begin a new database transaction (assuming we’re connected to a transactional database!)
99     db.transactionStart();
100 
101     // […] do stuff here
102 
103     if (saveChanges)
104         db.transactionCommit(); // either commit the current transaction…
105     else
106         db.transactionRollback(); // …or roll it back
107     ---
108 
109 
110     ### Prepared Statements
111 
112     Most database access in $(I oceandrift) happens through Prepared Statements.
113 
114     Prepared Statements are compiled before execution
115     and allow the use of dynamic parameters (see: [Stattement.bind]).
116     This implies there’s no need for manual quoting or escaping with regard to dynamic parameters.
117     In such a prepared statement actual values are substituted by “placeholders” (usually in form of a question mark `?` character).
118 
119     $(TIP
120         Reuse of prepared statements (for multiple executions) can improve performance.
121         This is because the SQL query needs to parsed only once
122         and the database engine can also reuse the query plan it determined etc.
123     )
124 
125     Unlike in many other database libraries there’s no “vanilla” `.query` function.
126     This also means there’s no need for a function to escape SQL string literals.
127 
128 
129 
130 
131     ## Driver Developer’s Guide
132 
133     $(NOTE
134         Feel free to skip this chapters if you don’t plan on implementing your own database driver(s).
135     )
136 
137     In order to add support for another database implementation/engine/whatsoever,
138     create a new driver struct or class matching the [DatabaseDriverSpec] interface.
139     Said interface only exists for documentation purposes.
140     There is no need to actually have your driver class implement (“inherit from”) [DatabaseDriverSpec].
141     It might become `private` in the future anyway.
142 
143     Connection setup should happen via its constructor,
144     but do not establish a connection yet;
145     that’s what the [DatabaseDriverSpec.connect] function/method is for.
146 
147     Provide the current connection status via a [DatabaseDriverSpec.connected] function/method.
148 
149     Clean connection shutdown happens through a call to [DatabaseDriverSpec.close].
150     Disconnect the underlying database connection there.
151     Beware a user may call [DatabaseDriver.connect] later of course.
152 
153     If the underlying database supports toggling “auto commit” mode,
154     this functionality should be provided via [DatabaseDriverSpec.autoCommit].
155     There’s also a corresponding getter method.
156 
157     Manual transaction control is done through:
158     $(LIST
159         * [DatabaseDriverSpec.transactionStart]
160             – starts/begins a new transaction
161 
162             might commit a currently active transaction, just stick to your database’s active defaults;
163             usually done by executing a SQL statement like `BEGIN TRANSACTION;`
164         * [DatabaseDriverSpec.transactionCommit]
165             – commit/save the current transaction
166 
167             usually done by executing a SQL statement like `COMMIT;`
168         * [DatabaseDriverSpec.transactionRollback]
169             – rollback/abort the current transaction
170 
171             usually done by executing a SQL statement like `ROLLBACK;`
172     )
173     For non-transactional databases you probably want to throw an Exception here.
174 
175     [DatabaseDriverSpec.execute] is a method that takes any SQL statement
176     that (usually) shouldn’t return any data (e.g. `CREATE TABLE …`)
177     and immediatly executes it.
178     Indicate errors by throwing an appropriate [Exception].
179     In other database libraries such functions would often return the number of affected rows or similar,
180     $(I oceandrift) currently defines this function as a `void` one; might be subject to change.
181 
182     [DatabaseDriverSpec.prepare]: compile the passed SQL statement and return a Prepared Statement.
183     Preparation (compilation) errors should be indicated by throwing an appropriate [Exception];
184     You must not return [null].
185 
186 
187     ### Prepared Statements (Driver)
188 
189     Prepared Statements are represented by instances of [Statement].
190     Create a class that implements said interface and wrap your database specific statement implementation.
191 
192     #### Dynamic Parameters (Driver)
193 
194     Dynamic parameters are set via the [Statement.bind] overloads.
195     If the underlying database clients expects zero(0)-indexed indices, substract `1` from the passed index number.
196     For datatypes your database engine doesn’t support, convert them to alternatives where appropriate.
197     Perhaps [string] can act as a last resort catch-all solution.
198 
199     #### Statement Exeuction (Driver)
200 
201     When [Statement.execute] is called, execute the prepared statement with the currently bound parameters.
202     Make the first result row available as `.front` if available (otherwise `.empty` should evaluate to [true]).
203 
204     $(PITFALL
205         Ensure Row does not point to any memory owned by the database driver.
206         Memory could get corrupted when references to it outlive the statement handle.
207         `.idup` is your friend.
208     )
209 
210     Statements form an $(B InputRange).
211     $(LIST
212         * [Statement.front]
213             – each range element represents an individual row
214         * [Statement.popFront] advances the cursor to the next row.
215         * [Statement.empty] indicates whether `.front` has no further row loaded.
216             For empty result sets this should be immediatly set to return [true].
217     )
218 
219     Regarding such query results:
220     $(LIST
221         * Convert column values to a compatible type of the [DBValue] tagged algebraic (“union”)
222         * [Row] is just a struct that wraps an array of [DBValue]s.
223             (This approach leads to better readable error messages than an plain alias.)
224     )
225 
226     Similar to [DatabaseDriverSpec] there’s a [Statement.close] method.
227     Finalize/close the underlying prepared statement and do any necessary cleanup.
228     If possible, code this with the assumption that your users could forget to call it – because they eventually will.
229 
230     $(NOTE
231         Special thanks to Paul “Snarwin” Backus.
232     )
233  +/
234 module oceandrift.db.dbal.driver;
235 
236 import std.meta : staticIndexOf;
237 import std.range : isInputRange;
238 import std.sumtype;
239 
240 public import std.datetime : Date, DateTime, TimeOfDay;
241 
242 @safe:
243 
244 /++
245     Database Driver API specification
246 
247     Specifies a unified interface for database client implementations.
248 
249     There is no reason to use this.
250     Exists for documentation purposes only.
251 
252     Feel free to implement drivers using structs.
253 
254     oceandrift is designed to work with anything that corresponds to the API shown here.
255  +/
256 interface DatabaseDriverSpec
257 {
258     @safe
259     {
260         /++
261             Establishes a fresh connection to the database
262          +/
263         void connect();
264 
265         /++
266             Shutdown the database current connection
267 
268             Idiomatically used with a scope guard:
269             ---
270             // DatabaseDriver db = …;
271             db.connect();
272             scope(exit) db.close();
273             ---
274          +/
275         void close();
276 
277         /++
278             Gets the current connection status
279 
280             Returns:
281                 true = connection active
282          +/
283         bool connected();
284     }
285 
286     @safe
287     {
288         /++
289             Determines whether “auto commit” is enabled for the current database connection
290          +/
291         bool autoCommit();
292 
293         /++
294             Enables/disables “auto commit”
295             (if available on the underlying database)
296          +/
297         void autoCommit(bool enable);
298 
299         /++
300             Begins a database transaction
301             (Requires support by the underlying database, of course)
302          +/
303         void transactionStart();
304 
305         /++
306             Commits the current transaction
307          +/
308         void transactionCommit();
309 
310         /++
311             Rolls back the current transaction
312          +/
313         void transactionRollback();
314     }
315 
316     @safe
317     {
318         /++
319             Executes the provided SQL statement
320 
321             Not intended to be used with querys.
322             Result does not get fetched (or is discarded immediatly).
323 
324             Params:
325                 sql = SQL statement to execute
326 
327             Throws:
328                 Exception on failure (specific Exception type(s) may vary from implementation to implementation)
329          +/
330         void execute(string sql);
331 
332         /++
333             Prepares the passed SQL statement
334 
335             Params:
336                 sql = SQL statement to prepare
337 
338             Throws:
339                 Exception on failure (specific Exception type(s) may vary from implementation to implementation)
340          +/
341         Statement prepare(string sql);
342 
343         ///
344         DBValue lastInsertID();
345     }
346 }
347 
348 /// TODO
349 enum bool isDatabaseDriver(T) = true;
350 
351 /++
352     Prepared Statement Handle
353  +/
354 interface Statement
355 {
356 @safe:
357     /++
358      +/
359     void close();
360 
361     /++
362         Binds the passed value to the specified dynamic parameter
363 
364         $(PITFALL
365             Dynamic paramters are 0-indexed,
366             i.e. the first parameter is `0` (not `1`).
367         )
368 
369         $(SIDEBAR
370             Yes, indexes start with 1 in SQL,
371             but in oceandrift they don’t.
372 
373             I’ve not only considered implementing them 1-indexed
374             (like PHP’s PDO or the SQLite3 C library does),
375             I even implemented them that way.
376             And got burned. Multiple times.
377 
378             In the end it’s a choice between two evils (as in: inconsistencies):
379             $(LIST
380                 * As we can’t change SQL and the database, those are always going to be 1-index.
381                 * Query results will always be 0-indexed,
382                   unless we’d wrap them into a magic type that would make them inconsistent with the rest of the D ecosystem
383                   (or use AAs – this option also comes with its own pitfalls and disadvantages).
384                 * Dynamic parameters are the only thing we can realistically choose to have either zero- or one-indexed.
385                   They can now be inconsistent with either query results or the database.
386             )
387 
388             Data structures in D are 0-indexed (like in most programming languages).
389             So I chose to go with this for dynamic parameters on the D side as well.
390 
391             $(BLOCKQUOTE
392                 In D (where 0-indexed arrays and slices are prefered) 0-indexing is the way to go whereever possible.
393             )
394 
395             $(TIP
396                 Oh well, even in PHP `PDO::FETCH_NUM` means result is an array starting at column 0.
397                 Inconsistent, isn’t it?
398             )
399         )
400 
401         Params:
402             index = 0-indexed index-number of the dynamic parameter to bind to
403             value = value to bind
404      +/
405     void bind(int index, const bool value);
406     /// ditto
407     void bind(int index, const byte value);
408     /// ditto
409     void bind(int index, const ubyte value);
410     /// ditto
411     void bind(int index, const short value);
412     /// ditto
413     void bind(int index, const ushort value);
414     /// ditto
415     void bind(int index, const int value);
416     /// ditto
417     void bind(int index, const uint value);
418     /// ditto
419     void bind(int index, const long value);
420     /// ditto
421     void bind(int index, const ulong value);
422     /// ditto
423     void bind(int index, const double value);
424     /// ditto
425     void bind(int index, const const(ubyte)[] value);
426     /// ditto
427     void bind(int index, const string value);
428     /// ditto
429     void bind(int index, const DateTime value);
430     /// ditto
431     void bind(int index, const TimeOfDay value);
432     /// ditto
433     void bind(int index, const Date value);
434     /// ditto
435     void bind(int index, const typeof(null));
436 
437     /++
438         Executes the prepared statement using the currently bound values
439 
440         Access the result through the $(I InputRange) interface of the [Statement] type.
441         Check whether there are any rows via the `.empty` property.
442      +/
443     void execute();
444 
445     /++
446         Determines whether there are any rows loaded (or left).
447      +/
448     bool empty();
449 
450     /++
451         Advance to the next row
452 
453         Makes the next row available via `.front`.
454 
455         $(NOTE This is an $(I InputRange). Do not call `popFront()` on empty ranges.)
456      +/
457     void popFront();
458 
459     /++
460         Currently fetched (loaded) row
461 
462         $(NOTE This is an $(I InputRange). Do not call `front()` on empty ranges.)
463      +/
464     Row front();
465 }
466 
467 /++
468     Binds the passed value to the specified dynamic parameter
469 
470     Universal/driver-independent bind implementation for [DBValue].
471  +/
472 void bindDBValue(Statement stmt, int index, const DBValue value)
473 {
474     value.match!(
475         (ref const typeof(null) value) => stmt.bind(index, value),
476         (ref const bool value) => stmt.bind(index, value),
477         (ref const byte value) => stmt.bind(index, value),
478         (ref const ubyte value) => stmt.bind(index, value),
479         (ref const short value) => stmt.bind(index, value),
480         (ref const ushort value) => stmt.bind(index, value),
481         (ref const int value) => stmt.bind(index, value),
482         (ref const uint value) => stmt.bind(index, value),
483         (ref const long value) => stmt.bind(index, value),
484         (ref const ulong value) => stmt.bind(index, value),
485         (ref const double value) => stmt.bind(index, value),
486         (ref const const(ubyte)[] value) => stmt.bind(index, value),
487         (ref const string value) => stmt.bind(index, value),
488         (ref const DateTime value) => stmt.bind(index, value),
489         (ref const TimeOfDay value) => stmt.bind(index, value),
490         (ref const Date value) => stmt.bind(index, value),
491     );
492 }
493 
494 static assert(isInputRange!Statement);
495 
496 /++
497     Executes a «Prepared Statement» after binding the specified parameters to it
498 
499     $(TIP
500         Best used with $(B UFCS):
501 
502         ---
503         stmt.executeWith(param1, param2);
504         ---
505     )
506 
507     $(SIDEBAR
508         UFCS is also the reason why this isn’t named just `execute`.
509         It wouldn’t work that way because [Statement] already defines a method with said name.
510 
511         Moving this function into the Statement interface would bloat it for no good reason.
512     )
513 
514     $(WARNING
515         If called with less arguments (excluding the statement itself, obviously) than the Statement has dynamic parameters,
516         the rest of the parameters will retain their current values.
517 
518         This might lead to unexpected results. At least, it’s bad pratice.
519 
520         Future versions may check for this and error when there’s a mismatch in paramter count.
521     )
522  +/
523 void executeWith(Args...)(Statement stmt, Args bindValues)
524 {
525     foreach (int idx, val; bindValues)
526     {
527         stmt.bind(idx, val);
528     }
529 
530     stmt.execute();
531 }
532 
533 /++
534     Database Value (representing a row’s column)
535  +/
536 alias DBValue = SumType!(
537     typeof(null),
538     bool,
539     byte,
540     ubyte,
541     short,
542     ushort,
543     int,
544     uint,
545     long,
546     ulong,
547     double,
548     const(ubyte)[],
549     string,
550     DateTime,
551     TimeOfDay,
552     Date,
553 );
554 
555 alias null_t = typeof(null);
556 
557 /++
558     Returns the value stored in the DBValue sumtype
559 
560     Throws:
561         [std.sumtype.MatchException] if the DBValue doesn’t hold the specified type T
562 
563     See_Also:
564         [getAs]
565 +/
566 T get(T)(DBValue value)
567 {
568     return value.tryMatch!((T t) => t);
569 }
570 
571 /++
572     Determines whether a DBValue is `null`
573 
574     Returns:
575         true = if NULL
576 +/
577 bool isNull(DBValue value)
578 {
579     try
580     {
581         return value.tryMatch!((typeof(null)) => true);
582     }
583     catch (MatchException)
584     {
585         return false;
586     }
587 }
588 
589 /++
590     Returns the value stored in the DBValue
591     casted (or converted) to the specified type `T` if possible
592 
593     Throws:
594         MatchException on failure
595  +/
596 T getAs(T)(DBValue value) pure
597 {
598     static if (is(T == bool))
599         return value.tryMatch!(
600             (ref const bool v) => v,
601             (ref const byte v) => cast(bool) v,
602             (ref const ubyte v) => cast(bool) v,
603             (ref const short v) => cast(bool) v,
604             (ref const ushort v) => cast(bool) v,
605             (ref const int v) => cast(bool) v,
606             (ref const uint v) => cast(bool) v,
607             (ref const long v) => cast(bool) v,
608             (ref const ulong v) => cast(bool) v,
609         );
610 
611     else static if (is(T == ubyte))
612         return value.tryMatch!(
613             (ref const bool v) => cast(ubyte) v,
614             (ref const byte v) => cast(ubyte) v,
615             (ref const ubyte v) => v,
616             (ref const short v) => cast(ubyte) v,
617             (ref const ushort v) => cast(ubyte) v,
618             (ref const int v) => cast(ubyte) v,
619             (ref const uint v) => cast(ubyte) v,
620             (ref const long v) => cast(ubyte) v,
621             (ref const ulong v) => cast(ubyte) v,
622         );
623 
624     else static if (is(T == int))
625         return value.tryMatch!(
626             (ref const bool v) => cast(int) v,
627             (ref const byte v) => cast(int) v,
628             (ref const ubyte v) => cast(int) v,
629             (ref const short v) => cast(int) v,
630             (ref const ushort v) => cast(int) v,
631             (ref const int v) => v,
632             (ref const uint v) => cast(int) v,
633             (ref const long v) => cast(int) v,
634             (ref const ulong v) => cast(int) v,
635         );
636 
637     else static if (is(T == uint))
638         return value.tryMatch!(
639             (ref const bool v) => cast(uint) v,
640             (ref const byte v) => cast(uint) v,
641             (ref const ubyte v) => cast(uint) v,
642             (ref const short v) => cast(uint) v,
643             (ref const ushort v) => cast(uint) v,
644             (ref const int v) => cast(uint) v,
645             (ref const uint v) => v,
646             (ref const long v) => cast(uint) v,
647             (ref const ulong v) => cast(uint) v,
648         );
649 
650     else static if (is(T == long))
651         return value.tryMatch!(
652             (ref const bool v) => cast(long) v,
653             (ref const byte v) => cast(long) v,
654             (ref const ubyte v) => cast(long) v,
655             (ref const short v) => cast(long) v,
656             (ref const ushort v) => cast(long) v,
657             (ref const int v) => cast(long) v,
658             (ref const uint v) => cast(long) v,
659             (ref const long v) => v,
660             (ref const ulong v) => cast(long) v,
661         );
662 
663     else static if (is(T == ulong))
664         return value.tryMatch!(
665             (ref const bool v) => ulong(v),
666             (ref const byte v) => ulong(v),
667             (ref const ubyte v) => ulong(v),
668             (ref const short v) => ulong(v),
669             (ref const ushort v) => ulong(v),
670             (ref const int v) => ulong(v),
671             (ref const uint v) => ulong(v),
672             (ref const long v) => ulong(v),
673             (ref const ulong v) => v,
674         );
675 
676     else static if (is(T == string))
677         return value.tryMatch!(
678             (ref const string v) => v,
679             (ref const const(ubyte)[] v) => cast(string) v.idup,
680         );
681 
682     else static if (is(T == Date))
683         return value.tryMatch!(
684             (ref const Date v) => v,
685             (ref const DateTime v) => v.date,
686             (ref const string v) => Date.fromISOExtString(v),
687         );
688 
689     else static if (is(T == TimeOfDay))
690         return value.tryMatch!(
691             (ref const TimeOfDay v) => v,
692             (ref const DateTime v) => v.timeOfDay,
693             (ref const string v) => TimeOfDay.fromISOExtString(v),
694         );
695 
696     else
697         static assert(0, "No getAs!T() routine for T == " ~ T.stringof);
698 }
699 
700 enum bool isInSumType(T, SumType) = (staticIndexOf!(T, SumType.Types) >= 0);
701 
702 enum bool isDBValueCompatible(T) = isInSumType!(T, DBValue);
703 
704 /++
705     Database Result Row
706  +/
707 struct Row
708 {
709     DBValue[] _value;
710     alias _value this;
711 }