Pull user data from custom table into list

Several add-ons have been developed to extract data created by third party plugins and not stored in the wordpress user and user meta tables.

Request an add-on (charges apply) or If you some php skills, use the example plugin to write your own addon.

If you have a custom table that has user data in it, you can pull it into a list by writing a little site specific plugin and making use of some of the hooks available in the amr-users plugin.

The hooks you may need are:

Read the filter documentation and see the add-on plugins to see how to use these hooks.  Many of the add-ons use them to add data from a variety of different styles of tables.

See https://wpusersplugin.com/2940/filters/#examples

More details

See one of the add-on plugins or thee xample site specific plugin.  Do NOT cut and paste the code here. It is meant to inform only and will NOT  work as is.

 add_filter('amr_get_fields', 'amr_add_my_fields', 1);
 add_filter('amr_get_users_with_meta', 'amr_get_my_users_data', 1);
function amr_add_my_fields ($keys) {

// write code here to return the list of additional fields
// you'd like to see in your user lists. 
//IE these are fields in your custom table that are 
//NOT in wordpress user or usermeta table.

// the function will be passed the existing list of fields 
//and should add the additional fields to the array.  eg: 

$keys[] = 'newfield';

return ($keys);

// then return the array $keys

}

 

function amr_get_my_users_data ($users) {

// write code here to get the user data from the custom tables. Various approaches are open to you.  Either

  • drive from array of user passed to the function, or
  • from the array of user data from your custom table.. then add to the users array passed extradata = sql query to fetch data from custom table keyed by userid;
for each $extradata {

    $users[$userid]['newfield'] =  $extradata['newfield'];

// then return the array $users
    return ($users);

/pre>

Alternative (or to format nicely)

The function above has to process ALL the users.  The data will then be available in csv export.  If you only need the data to be displayed, then you could use a formatting function  rather.  The amr-users plugin will look for a custom formatting function for each column to be displayed for each user.
function ausers_format_newfield($value, $user) {

  $customdata_maybearray = code to get data ($user->ID);

  $html = implode (', ',$customdata_maybearray)

}

 

Cacheing (amr-users)

Why cache?

Cacheing is required because:

  • we want to display, query, filter and sort the data BUT
  • some plugins that create user data do it in ‘funny’ non wp standard ways
    • they stick all their user data in one user-meta record, sometimes as an object, sometimes as an array…. (eg: ym (yourmembers), although they appear to be moving to standard wp), so we cannot simply ‘select where meta = ‘
    • they store their user data in non wp tables, so that data has to be fetched separately.  This also means that the wp user update actions don’t fire.  So any recaching dependent on that may  not happen.

So the plugin grabs, extracts, does some magic and stores the data in a separate generic cache table.  Originally this was aimed primary at csv extraction.

When to cache? (settings)

Cache settings

Ideally the lists should be up to date when you see them. BUT for reasons above this may not always be possible.  So consider the following scenarios:

Internal admin access only?

  • Consider switching off ALL auto caching.
  • Only rebuild the data when you view the list.

Public lists, high level of user updates?

  • Strongly suggest do NOT recache on every user update
  • Cache at the greatest possible interval (less demand on sql database) – maybe daily ?
  • Manual refresh can be made available via an icon link on user list page – see admin refresh link.

Public lists, very low level user updates?

  • you must be sure about what plugins are causing user or user meta updates.  If you have a plugin that is tracking the pages people view, or their logins and storing the data in the user meta… do I need to say more?  Re caching on every user update would place a huge sql demand on your system.
  • Consider caching on use update only – no scheduled caching.

How does the auto cache rebuild work?

(version 3.4.9 at time of writing)

The cache can be set to automatically update in the ‘background’.  This is achieved using standard wordpress cron features.  If you are new to cron, then basically all you need to know is that it needs some activity on your website to run.  eg: a page request.  A scheduled job will only run on the first page request after the jobs scheduled time.  If you have a very low traffic site, this could be quite late.

A cache update request is initiated via cron on the following wordpress actions:

  •     add_action(‘profile_update’,’amr_user_change’);
  •     add_action(‘user_register’,’amr_user_change’);
  •     add_action(‘deleted_user’,’amr_user_change’); // also for wpmu
  •     add_action(‘added_user_meta’,’amr_user_meta_change’);
  •     add_action(‘updated_user_meta’,’amr_user_meta_change’);
  •     add_action(‘deleted_user_meta’,’amr_user_meta_change’);
  •     add_action(‘make_spam_user’,’amr_user_meta_change’);
  •     add_action(‘make_ham_user’,’amr_user_meta_change’);
  •     add_action(‘remove_user_from_blog’,’amr_user_change’);
  •     add_action(‘add_user_to_blog’,’amr_user_change’);

If your install has a user update that does NOT trigger a wordpress action (eg: another plugin stores user data somewhere else), then the plugin has no way of knowing that a user update has happened.

If the other plugin uses its own update actions, you could add a custom function call on that action similar to the above calls.

Alternatively either a regularly scheduled update or a manual on demand refresh may be adequate.

The cache update request attempts to schedule a master cron job.  Many of these wordpress actions may trigger within milliseconds of each other.   The plugin therefore checks whether a cache update is scheduled within the near future to avoid duplicate work.

Master cache cron job shown in amr cron manager

Master cache cron job shown in amr cron manager

The master cron cache job then schedules individual jobs for each of the reports.  They are scheduled to run separately, not concurrently.  In your cron manager plugin you should see an entry for each job.

When testing, you should therefore do a page request or two approximately minutes apart for each report that you have designed.  Check the cache log in between.

A cron run for each report

Cache storage

(version 3.4.9 at time of writing)

This may change at a later stage for installs that only  use standard wordpress user meta.  Other installs will need the current setupo to deal with their user data created by other plugins.

Currently a generic cache report table in csv format is created.  This made sense originally, however as the plugin became popular more features were requested.  This approach is not ideal for optimum efficiency (especially large high traffic installs).  If one is only using standard wordpress user and user meta data, then the lists could be more efficient.   To cope with non standard data, the curremt approach will have to continue.

Cache Log

The cache log will list the most recent cache rebuild activities.  If you are concerned or not sure how the cache updates are initiated, please pay close attention to the cache log when testing.  For more details see cacheing-the-cache-log

User Meta Update causes cron cache rebuild

Cache Status

The cache status page lists

  • the size (lines) of a list
  • when the cached list was last built
  • how long it took to build the cache
  • the peak php memory requirement while running the cache update
  • some info on the list

It also has options to totally clear out the various cache records (in case your install gets in a knot), and force the plugin to start again.

Cache Status

Cacheing – the cache log

The amr users cache log is a useful tool especially when using background (cron) cacheing.

Cache log example

Cache log example, where user master was updated (not meta). A Cache rebuild had been scheduled, but not yet run.

 The cron log

  • shows key actions with latest at the top, so read from the bottom up.
  • will show more messages if wp-debug is enabled
  • does not keep a lot of history – it cleans up after itself.  It’s a testing tool primarily.
  • will indicate runtimes
  • will indicate batches of updates of the cache records.
  • will list why a cache was initiated
    • master user update
    • user meta update
    • background request,
    • else  manual request

Example Screenshots

User meta update request:

  • the green text showing the action that causes a request to rebuild cache
  • the bolded text controlling background job that setups the cron jobs for each of the lists
  • the italic text showing the completion of the cache rebuilds for each list.

User Meta Update causes cron cache rebuild

Change of settings and background run

  • the change of setting is listed and the time that the next cron job will run
  • the rebuilds of reports 1 and 2 have been scheduled but not yet run

Background request scheduled, preceded by some manual request, also some debug statements on memory

Cron jobs have been run for reports

An update of the previous image - the jobs have now been run.

An update of the previous image – the jobs have now been run.

 

Why use the cron log?

One can confuse oneself mightily if

  • changing settings,  while background jobs are running, or
  • on very large data bases if runs take a while
  • if you have user update set, but no updates seem to happen (maybe your user updates are not actually to the wp user meta.)

So take it easy – use the tools available.  A good cron manager plugin may also be helpful. There are many around. I like my cron manager plugin because you can reschedule any cron job, eg make them run immediately.  It’s included in developer memberships.

Cacheing – public on demand refresh

Allow users to request a rebuild of the amr users plugin data?

On demand refresh setting

On demand refresh setting

Setting the refresh setting for a list will cause a refresh icon to show below the list.

Use this cautiously and only if it is imperative that users be able to request a rebuild of the cache.

Refresh icon shown on public list

Refresh icon shown on public list

Customise the refresh link

In the general settings, you can specify what the refresh link should look like and the hover text.   Plain text or image link – it’s your choice.

refresh setting

refresh text or image setting

Code your own refresh link

It’s just html folks – a glorified hyperlink with a url query parameter  You can also add your own link and insert it into the page content below the shortcode.

Link from user directory to user detail or profile

This is now very simply achieved within one page using the amr-users-plus plugin.    Or if you have special requirements, that is all possible too.

Requirements:

OR

  • a user list (from amr-users plugin)
  • a profile page (from another plugin or by wordpress)
  • or another amr users list ?
  • (optional) your own site specific plugin – lf linking to another amr-users lists, or linking to some other profile page that has not yet been allowed for in the link types.

The user list or directory

The directory part is easy – choose or configure a user list.  Set up the list first.  Then later, when you know which profile or detail method you are going to use, you will come back to add the detail link to a field in this list.

The detail or profile page

There are many choices here.  For example:

  1. a wordpress user archive  or
  2. another plugin’s profile page (the profile page must accept a user parameter (eg userid) so that one can generate html that will link to the profile page.
  3. maybe another more detailed user list (requires extra coding on your part)

WordPress user archive  or another plugin’s profile page (bbpress, buddypress etc)

These options are provided by default in amr-users.

Configure your list:

  1. Look for the linktype column
  2. Choose a suitable field to hold the link
  3. Click the dropdown list and Choose the type of link you want, the list may depend on what plugins you have active. eg:
    1. edit user page
    2. wordpress author archive
    3. bbpress profile
    4. buddypress profile
    5. user’s url
    6. mailto link
    7. etc
  4. update, rebuild cache, test.

Configuring links for fields in the user reporting

Solutions requiring site specific plugins that use amr-users filters:

Link to a more detailed user list example.

This option requires you to do some coding.

  • A ‘linktype’ to link to the detailed list needs to be created.
  • the html for the links needs to be generated when that linktype has been specified

See an example here: This http://directories.wpusersplugin.com/simple-user-list/ list, click a user, it should link to http://directories.wpusersplugin.com/memberdetails/?filter=1&ID=21.

Simple Directory

should link to:

Detail showing single user

 

This option is site specific and thus needs some php coding skills.   You could base your code on this example :

https://wpusersplugin.com/related-plugins/amr-user-plugin-add-ons/example-site-specific-add-on-using-filters/

Download the code example and look at it.  It should be fairly clear what to do if you have some knowledge of php and wordpress filters.

At the very least, you may want may to change the line that says

        home_url('/memberdetails/')  /* replace "/memberdetails/" with the slug of your user details page  */

to

        home_url('/yourmemberdetailsslug/')

where “your memberdetails slug” is the slug of the page that has a user list that you are using for your member deatils. Also have a look at the commented out lines 72 and 73. Uncomment if you want to show the current user as a default on that page.
If you need the lists to be created and the code edited  for you, please see http://webdesign.anmari.com/about/fees/

Export wordpress users to csv using amr users plugin

The “amr users” plugin allows one to create a wide variety of user lists accessing as little or as much of the user data as you like. One can then export these views for further analysis and manipulation.

There are several ways to export users to csv using amr-users

  • Public csv export by stored file on server
  • Non public real time csv export, or
  • Non public real time  csv as .txt export (for excel users, to force the excel text import wizard to appear)
  • Filtered CSV export if using amr-users-plus, by filtering by some fields and then click the ‘CSV filter’ button near the apply filter and clear filter buttons.

The overview screen where user lists can be managed

‘Public’ lists csv export

  • will generate an actual file that can be downloaded
  • a export icon will appear at the bottom of the list (both front pages and admin pages)
  • the list has been ‘ticked’ as public in the overview screen
  • you may or may not have it available on a front page in a shortcode (just because it is ticked public, does not mean that the general public can access it – it must also be in a shortcode in a page
  • the csv file location is obfuscated (hidden, but accessible)
  • you should use good sense in deciding what data should be made available on that list

Non Public Lists

Admin (non public) csv or txt export users options

  • In  order to protect the user data, non public lists are only written to csv at request time.
  • The ‘file’ is sent generated and sent direct to your browser.  No file is stored on the server.
  • This non-stored file option may have implications for people who have magic quotes enabled on their server.

Non Public csv export as .txt file

  • For people who use excel and have troubles with double quotes in csv.
  • Will export the csv list as a .txt file to force excel to use the text import wizard to allow control over the import options

See

amr users css, html and styling

How to change the look of your user list:

(see screenshots further down)

CSS:

  • Plugin comes with a default css file used mainly for the public lists
  • Default css tries to let your theme do most of the styling for consistent appearance and just add essential extras
  • Default css can be switched off completely
  • There are many css classes and selectors so that you can target almost any part of the list to achieve your own styling.  Please use your browser’s ‘inspect element’ to investigate the classes for your data and lists.
  • Default css can be activated for certain pages only
  • If you need custom css – maybe because you’ve created a very wide list or your data has forced the table to be wide, see adding custom css to a wordpress website

HTML:

  • In version 3.4 onwards, the admin or backend lists all use table html as these lists are also used to administer the users
  • Tables can be good and tables can be tricky.  Since YOU are determining the columns, the field values in them and the size of the table, the plugin should not attempt to impose any fixed sizing.   This means if you have a wide list, you may need to add some css to make it behave the way you want.
  • For ‘public’ lists, a ‘public html type‘ can be set to use simple html which allows more flexible styling of the lists.  The same “classes” will be used so much css can be shared for consistency of appearance.
  • so public html type can be:
    • table – yes a normal html table
    • simple – a html5 non table format

Images:

  • csv export image link and refresh link icons or text can be totally customised (general settings)
  • default avatar type (used if user does not have their own avatar) will be set by your wordpress settings (settings > Discussion > scroll down to set default avatar )
  • avatar size can be set as a default for all lists (general settings) and/or customised per list (configure list or overview)

CSS and image settings screen:

Css settings for ALL lists – in the General settings

 

Setting the public html type:

Set the public html type for a list in the overview settings

Css Selectors available

  • Each user record or row has a vcard class, and an even or odd class
  • Each field for that user has it’s fieldname as a class
Css selectors available for a demo page at https://bp.wpusersplugin.com/user-list-files-urls/
Css selectors available in the non-table html

Multiple column user list in wordpress

How to create a multiple column member directory in wordpress, using the free amr-users plugin.

Member directory with custom avatar and no table html

Member directory with custom avatar and no table html

Key aspects:

  • non table html
  • css to set the width for each user div and to ‘float’ the divs next to each other

Steps:

  1. activate plugin , start with one of the sample lists
  2. make the list “public”  ( the backend admin lists all use table html )
  3. make the list use the “simple” (ie: non table html )
  4. create a page, enter short code [userlist list=n]  where ‘n’ is the number of your public list.
  5. view the page.  This should use the default css.  If your page width is wide enough, the users should float up next to each other.   If not, you will need to adjust the css.

 amr users “configure the list” settings

Public Html Settings

Public Html Settings – set the tyep to “simple” and make the list “public”

Using firebug to inspect the css:

Default css to set the width and float the user records up next to each other

Default css to set the width and float the user records up next to each other

 

Getting started with amr-users

On activation

When you activate the plugin:

  • there will be some sample reports using the standard wordpress fields
  • cache tables will be created

What to do next: Summary

Create sample users that have data

  1. Create sample users with sample data.
    1. create wordpress users and if using any other plugins, do whatever actions you need to create sample data (eg: login/out, download something, create a post as that user etc)
  2. There must be at least one entry for each field you want to see.  Just designing a field in whatever plugin you are using is not enough. There must be at least one user with data for that field!

Find the fields

  1. Execute “find the fields” to detect the data you have created.Find fields created by other plugins
  2. Rename anything that has an ‘ugly’ name (optional)Rename fields

Configure the lists

  1. Inspect the overview so you get a general understandingOverview  of lists and general settings
  2. Configure the lists

    Configure each individual list, add fields

    1. Add fields by entering a number for a display order – any number
    2. To select data by a field value (and exclude users who do not have that value), enter the selection values in the ‘include’ fields (comma separate multiple values).
    3. To exclude data by a field value, enter the exclusion values in the ‘exclude’ fields.  Users who have those values will NOT be listed.  Note the sanple lists usually exclude admin users and user id = 1  as example.
    4. Click Update field settings
  3. Rebuild the cache
  4. View the list

Cache Settings

Large or Live Sites:- Set your cache settings carefully

Do you have frequent user updates ?

Do NOT recache on every user update.  User updates include usermeta updates and could happen more often than you think (eg: got a login logger ?)

User meta data NOT stored in wpuser meta table

(eg: Cimy plugin and some others).  User updates may NOT trigger the wp user action.  Must heva batched re cacheing then.

Cache settings
If you make any changes to your system such as:

  • Add user plugins ?
  • Add fields ?
  • Add data ? : 

then Cache must be updated for changes to show. (duh!)

Problems?

  • have you worked through the steps above?
  • do you understand cron and cacheing
  • Look at ‘test your db’
  • do a search on this site with various keywords
  • look at other posts in this category
  • try the wordpress forum and this forum

 

Filters and actions (for developers)

amr-users version 3.2 upwards has wordpress style filters to allow adding in data from other sources.  I use  these myself for the add-on plugins

amr_get_fields  – ($keys)

  • is passed a set of keys or fields that the plugin already knows about from wordpress user or usermet tables or other add-ons
  • your filtering function must return add to t he array of fields to be used when configuring reports and return the augmented array.
  • additional keys or fields will be visible in the nice names list when ‘find fields’ is run

amr_get_users  – ($users)

  • passes the filtering function an array of users with basic user data
  • your filtering function must  return an array of users
  • this is useful if you need to add users that are NOT in wordpress user table. EG: subscribe2 newsletter subscribers.

amr_get_users_with_meta – ($users)

  • passes the filtering function an array of users with basic user data,plus meta data. as required for that report
  • your filtering function must  return an array of users with all the necessary data
  • this is useful for adding in data for users  that is not stored in the wordpress user tables.  Example: cimy plugin data
  • also could be used in the data is stored weirdly.  It gives an opportunity to look at the meta data and work with it (result will be in csv output).  Please also consider using a formatting function as an alternative (result will not be in csv output, but may be more efficient as will only execute for displayed users.)

amr_users_format_value – ( $text, $fieldname, $originalvalue, $user)

  • passes the filtering function :
    • formatted text
    • internal fieldname (see nice names settings page)
    • original value unformatted as found in the database
    • the current user array
  • must return either the origonal text or your special formatting (do  not echo it)
  • working example is amr users plus 2member
  • NB: there is also the option for a special formatting function for your fields.

 amr-users-linktypes – ($linktypes array)

  • passes array of linktypes – name and description for dropdown selection function.. $linktypes[‘name’] = __(‘description’);
  • filtering function must  return an array of  linktypes
  • is passed
    • the chosen linktype when formatting a field (the linktype may be the one you setup above)
    • the user array in case your url is user dependent
    • the field on which the link is to be displayed, in case your url is field dependent
  • must return a url

amr_users_headings – ($cols,$icols,$ulist)

  • passes the filtering function
    • array of current headings
    • array of ‘technical’ column names
    • the list number
  • filtering function must  return an array of  headings

amr-users-export-csv -( ‘list_users’, $ulist)

  • change the capability required to export to csv

amr_users_csv_line – ( $line[‘csvcontent’])

  • modify the csv line before export

amr_users_apply_filter_html ($apply_filter_html)

  • change the ‘apply filter’ html

amr_filter_lables

  •     $lables = apply_filters (‘amr_filter_lables’, $lables, $col, $selected, $type);

amr_filter_values

  •     $values = apply_filters (‘amr_filter_values’, $values, $col, $selected, $type);

Actions

amr-add-criteria-to-list

  • runs before the list is executed or cache fetched
  • for example: allows one to force criteria into the request field  (eg: show only logged in user)
  • receives the list number

Example Applications

amr-users-plus-s2

This add on adds users from an external table:

  • ‘amr_get_fields’ to add the Subscribe2 subscriber table ip address field
  • ‘amr_get_users_with_meta’ to add the Subscribe2 subscribers to the user list.  It maps the subscribe2 data where possible to existing fields:
    • email => user_email
    • user_registered => date
    • user_status => ‘confirmed’ or ‘unconfirmed’
  • and adds the ip address field in for those subscribers

amr-users-plus-cimy

This add on adds additional data from an external table for existing users. i IE: makes the external data look like user meta data.

  • ‘amr_get_fields’ to add whatever fields have been defined in the cimy fields table
  • ‘amr_get_users_with_meta’ to add the cimy data to the existing users, linking the users by the user id