The MySQL Plugin API is a neat way to add functionality to the MySQL server. Any user of MySQL is using it: see the storage engine plugins listed by SHOW PLUGINS
. There are many types of plugins. For example, the do-whatever-you-want daemon plugins. Among the hottest things that appeared lately are plugins that add new interfaces to MySQL, such as the Memcache plugin of MySQL 5.6.
MySQL Client Server Protocol | Memcache Protocol |
---|---|
Port 3306 | Port 11211 |
| | Daemon Plugin |
MySQL |
However, to me a PHP guy writing an extension, assimilating arbitrary libraries and glueing them together is reason enough to into plugin hacking It is tempting to a plugin developer to reuse code of the system being extended. MySQL 5.6.7-rc has more than 2 million lines of code. There must be hot stuff among it! But, is it "public", is there a public API?
A look at the daemon plugin example
Writing a daemon plugin is straight forward: grab the source, cd
into the plugin
directory and copy the daemon plugin example. Edit the structure that describes the plugin. The manual has more details.
mysql_declare_plugin(daemon_example) { MYSQL_DAEMON_PLUGIN, &daemon_example_plugin, "daemon_example", "Brian Aker", "Daemon example, creates a heartbeat beat file in mysql-heartbeat.log", PLUGIN_LICENSE_GPL, daemon_example_plugin_init, /* Plugin Init */ daemon_example_plugin_deinit, /* Plugin Deinit */ 0x0100 /* 1.0 */, NULL, /* status variables */ NULL, /* system variables */ NULL, /* config options */ 0, /* flags */ } mysql_declare_plugin_end;
Among others, the structure lists an init (daemon_example_plugin_init) and a deinit (daemon_example_plugin_deinit) function of the plugin. The init function is called when the plugin is loaded into the server during startup or using INSTALL PLUGIN
.
King Brian’s init function is below. It opens a log file and starts a heartbeat thread to fill the log.
static int daemon_example_plugin_init(void *p) {
DBUG_ENTER("daemon_example_plugin_init");
struct mysql_heartbeat_context *con;
pthread_attr_t attr; /* Thread attributes */
char heartbeat_filename[FN_REFLEN];
char buffer[HEART_STRING_BUFFER];
time_t result= time(NULL);
struct tm tm_tmp;
struct st_plugin_int *plugin= (struct st_plugin_int *)p;
con= (struct mysql_heartbeat_context *)
my_malloc(sizeof(struct mysql_heartbeat_context), MYF(0));
fn_format(heartbeat_filename, "mysql-heartbeat", "", ".log",
MY_REPLACE_EXT | MY_UNPACK_FILENAME);
unlink(heartbeat_filename);
con->heartbeat_file= my_open(heartbeat_filename, O_CREAT|O_RDWR, MYF(0));
/*
No threads exist at this point in time, so this is thread safe.
*/
localtime_r(&result, &tm_tmp);
my_snprintf(buffer, sizeof(buffer),
"Starting up at %02d%02d%02d %2d:%02d:%02d\n",
tm_tmp.tm_year % 100, tm_tmp.tm_mon+1,
tm_tmp.tm_mday, tm_tmp.tm_hour,
tm_tmp.tm_min, tm_tmp.tm_sec);
my_write(con->heartbeat_file, (uchar*) buffer, strlen(buffer), MYF(0));
pthread_attr_init(&attr);
pthread_attr_setdetachstate(&attr,
PTHREAD_CREATE_JOINABLE);
/* now create the thread */
if (pthread_create(&con->heartbeat_thread, &attr, mysql_heartbeat,
(void *)con) != 0) {
fprintf(stderr,"Could not create heartbeat thread!\n");
exit(0);
}
plugin->data= (void *)con;
DBUG_RETURN(0);
}
I have tried to highlight some functions and macros used: DEBUG_ENTER
, my_malloc
, my_open
, my_snprintf
, my_write
, DBUG_RETURN
. Obviously, the daemon plugin has to include some header files to get access to them. And, in daemon_example.cc you find the corresponding include statements.
DBUG_ENTER, DBUG_RETURN | include/my_dbug.h |
my_malloc | include/my_sys.h (mysys/my_malloc.c) |
my_snprintf | sql/sql_plugin_services.h |
my_write | include/my_sys.h (mysys/my_write.c) |
#include <my_global.h>
#include <sql_priv.h>
#include <stdlib.h>
#include <ctype.h>
#include <mysql_version.h>
#include <mysql/plugin.h>
#include <my_dir.h>
#include "my_pthread.h" // pthread_handler_t
#include "my_sys.h" // my_write, my_malloc
#include "m_string.h" // strlen
#include "sql_plugin.h" // st_plugin_in
Cool, plugin developers can use all of the servers’ code?!
Looking at the above list of included files one may come to the conclusion a plugin developer is allowed use any server code. But, then there is sql/sql_plugin_services.h
, which is included by sql_plugin.h
. Is this the formal public API for accessing server code. Walking in the dark with no manual or book guidance…
In sql_plugin_services.h
a couple of services are defined. The daemon example plugin is using the my_snprintf_service
.
static struct st_service_ref list_of_services[]= { { "my_snprintf_service", VERSION_my_snprintf, &my_snprintf_handler }, { "thd_alloc_service", VERSION_thd_alloc, &thd_alloc_handler }, { "thd_wait_service", VERSION_thd_wait, &thd_wait_handler }, { "my_thread_scheduler_service", VERSION_my_thread_scheduler, &my_thread_scheduler_handler }, { "my_plugin_log_service", VERSION_my_plugin_log, &my_plugin_log_handler }, { "mysql_string_service", VERSION_mysql_string, &mysql_string_handler }, }
Not recalling a note about any services in the manual, the happy grepping through the code continues. An enlightening code comment can be found in sql/plugin.cc
. Plugins are provided with "built-in functions".
/* link the services in */ for (i= 0; i < array_elements(list_of_services); i++) { if ((sym= dlsym(plugin_dl.handle, list_of_services[i].name))) { uint ver= (uint)(intptr)*(void**)sym; if (ver > list_of_services[i].version || (ver >> 8) < (list_of_services[i].version >> 8)) { char buf[MYSQL_ERRMSG_SIZE]; my_snprintf(buf, sizeof(buf), "service '%s' interface version mismatch", list_of_services[i].name); report_error(report, ER_CANT_OPEN_LIBRARY, dlpath, 0, buf); DBUG_RETURN(0); } *(void**)sym= list_of_services[i].service; } }
Internal APIs vs. public stable APIs for plugin developers
The official daemon example plugin is accessing server functionality in two ways: using services and direclty including whatever header is needed.
The key idea behind the service is probably the versioning. Plugin developers should be allowed to assume that version 1 of service A acts in a defined way, no matter what the MySQL server version is. On the contrary, my_write
could change its interface in the next server version and break a plugin without prior notice. However, there seems to be no registry for servies, no directory and not much of an API for a plugin developer to request the use of selected services of a certain version. Thus, this kind of service is nice but at the end of the day…?
The do-whatever-you-like style
The daemon example plugin is using a mixture of internal server APIs and plugin services. The few, internal my_sys.h
functions used are peanuts compared to the use of internal MySQL APIs by storage engine plugins. Note, that an API includes functions and data types. Given that, plugin developers don’t have much of a choice but to use internal APIs that may change at any time without prior notice.
If I was to write a plugin, I would bite the bullet and use any internal API, possibly, through a lightweight wrapper to minimize dependencies.
Happy hacking!
2012/11/08 at 22:12
Reaction on Facebook
—————
Anyone volunteering to write a MySQL hacking cookbook with little, easy to use code snippets? http://blog.ulf-wendel.de/2012/mysql-plugin-development-public-api/
MySQL Plugin development: public API? – Ulf Wendel
blog.ulf-wendel.de
MySQL Plugin development: public API?2012/11/08by admin|0 commentsTheMySQL Plugin APIis a neat way to add functionality to the MySQL server. Any user of MySQL is using it: see the storage engine plugins listed bySHOW PLUGINS. There are many types of plugins. For example, the do-whatever-you-wantdeam…
1
Gefällt mir · · Beitrag nicht mehr folgen · Teilen
Nils Hitze gefällt das.
Antony Dovgal Instead of code snippets (which are quite far from the real life usually) I’d prefer to have a
set of links to the plugins/engines. In that case, if I would be writing a plugin from scratch, it would be easier to look into the sources and see how other people solved similar problems.
vor 6 Stunden · Gefällt mir
Antony Dovgal But to be honest, the only problem that concerns me at the moment is that there isn’t a
decent plugin API. In fact, you can’t even call them plugins, they’re more like patches, ’cause a plugin requires full sources in order to be built.
vor 6 Stunden · Gefällt mir
Antony T Curtis There was a roadmap towards fully built-outside-source-tree plugins but development of
plugin api kinda stalled after I left.
I’m sorry.
vor 3 Stunden · Gefällt mir
Antony Dovgal Personally I don’t need an API, I just need all the headers installed, which extremely easy to
do.
vor 3 Stunden via Handy · Gefällt mir
Vladislav Vaintroub Download source distribution => get all headers.Extremely easy to do
vor 2 Stunden · Gefällt mir
Vladislav Vaintroub And Ulf, it is called “daemon”, not “deamon”
😉
vor 2 Stunden · Gefällt mir
Ulf Wendel The get all headers approach has two obvious downsides. I still have no examples using any of
the MySQL goodies that must be in those 2,000,00 loc. And, as a plugin writer, I have no way to argue if anyone says “this is internal, don’t use, we may change at any time…”
vor 2 Stunden · Gefällt mir
Ulf Wendel Vladislav Vaintroub plugins seem evil…
vor 2 Stunden · Bearbeitet · Gefällt mir
Vladislav Vaintroub as a plugin writer, please accept “this is internal, don’t use”. Otherwise the very next
thing you’ll ask would be nonsensical, like “how to make DBUG work on optimized executable”. I recall was a request like this in the past.
vor 2 Stunden · Gefällt mir
Antony Dovgal @Vladislav Vaintroub, are you serious?
Ever heard of -devel packages? Headers in /usr/include? No? Now try to imagine that mysql can be built with different options and the end user somehow has to learn what those options were in order to build a working plugin using the sources configured with exactly the same options. Ok, you were kidding, I got it.
vor 2 Stunden via Handy · Gefällt mir
Vladislav Vaintroub I’m serious. -devel packages for MySQL contain client, like they should. Server plugins
are hack, that is not suitable for packaging, in its current form. there is no API, storage engines use half of the server, mostly undocumented stuff.
vor 2 Stunden · Gefällt mir
Ulf Wendel Vladislav Vaintroub s/deamon/daemon – thx. Well, if there was any documentation about what is
internal and what not, I would follow it… are you saying plugin services is the only public, non-internal stuff? If so, why does the manual show internal DBUG stuff used in example code that, at least I see it that way, should be a blueprint for users.
vor 2 Stunden · Gefällt mir
Ulf Wendel ACK on hack.
vor 2 Stunden · Gefällt mir
Ulf Wendel Or, let’s not say hack but somewhat half-baken, unfinished.
vor 2 Stunden · Gefällt mir
Vladislav Vaintroub I have a pretty good I think criteria on when plugins are not hack. This will happen when
shared libraries stop linking with the server, on Windows or on Mac. Or would not have undefined symbols, on Linux. I.e plugins should not use any of server entry-points directly. There was related worklong, “server services”, started by Serg, which would, when finished, define that whole server API that plugins can use.
vor 2 Stunden · Gefällt mir
Ulf Wendel … which is certainly still a dream. Including versioning. And, maybe, one fine day, the importance
of unit tests and documentation will be recognized as well. However, dreaming, this is a huge piece of code we talk about. Legacy code. No blame on anybody that manages to survive it on a day to day basis.
vor 2 Stunden · Gefällt mir
Antony Dovgal I don’t quite care if it looks like a hack or something else to you. My point is that the current
situation is unacceptable and nobody is able to show me any real solution to it, except the boring whining regarding the complexity of the task (this kind of show continues almost 4 years already, btw). So here: install all the freaking server headers and be done with it. I’m ready to update my patch that does this.
vor 39 Minuten via Handy · Gefällt mir
Vladislav Vaintroub You need to take care of your tone, Anthony Dovgal. If you continue to demand a
solution in a harsh tone, like this, for a problem that is not obvious to anyone else but you, you’ll get exactly what you paid for. If you want to be heard, my sincere recommendation would be to humbly ask (humbly, as in “feature request”) on the bugdb. Luckily it is not me anymore, who has to deal with nonsensical feature request like this.
vor 29 Minuten · Gefällt mir
Ulf Wendel I understand the emotions. It has not been my goal to blame anybody by picking a random and
tiny aspect out of a 2,000,000 loc monster. Keeping this beast working is a fantastic achievement. I tried to take the position of a community plugin developer….Mehr anzeigen
vor 16 Minuten · Gefällt mir