Tweaking Applying std::tuple To Functors Efficiently

Overview

In previous posts I developed and tested code that could apply tuples to functions and function objects (functors) efficiently. In a reply comment to my last post, it was noted that apply_tuple() does not work with lvalues. This post outlines how to fix the code so it works with both lvalues and rvalues.


Some Code Demonstrating That Lvalues Won't Compile

Nothing fancy has to be done to show that passing an lvalue to apply_tuple() will fail to compile. One need only write some code like this:

// place apply_tuple-related code here

int add(int a, int b)
{
  return a + b;
}

int main()
{
  std::tuple<int,int> t;
  apply_tuple(add, t);
  // etc.
}

This code fails to compile since the tuple argument is not an rvalue and the tuple argument type is not an exactly-as-passed template type parameter.

C++11 defines a special template argument deduction rule when a function template accepts an argument as an rvalue reference to a template type. When this occurs, if the argument is an lvalue reference of type T, then it is treated as T&; if the argument is an rvalue reference of type T, then it is treated as T&&. This convenience is only applied when T is an exactly-as-passed template argument.

Note the tuple type is not exactly-as-passed (it is computed):

template <
  typename Op,
  typename... OpArgs,
  typename Indices = typename make_indices<0, OpArgs...>::type,
  template <typename...> class T
>
auto apply_tuple(Op&& op, T<OpArgs...>&& t)
  -> typename std::result_of<Op(OpArgs...)>::type
{
  return apply_tuple_impl<Indices>::apply_tuple(
    std::forward<Op>(op),
    std::forward<T<OpArgs...>>(t)
  );
}


Fixing The Problem

When such situations arise the easiest way to fix it is to add an overload of the function that accepts an rvalue reference so that it accepts an lvalue reference, i.e., use a const&, instead:

// Rvalue apply_tuple()...
template <
  typename Op,
  typename... OpArgs,
  typename Indices = typename make_indices<0, OpArgs...>::type,
  template <typename...> class T
>
auto apply_tuple(Op&& op, T<OpArgs...>&& t)
  -> typename std::result_of<Op(OpArgs...)>::type
{
  return apply_tuple_impl<Indices>::apply_tuple(
    std::forward<Op>(op),
    std::forward<T<OpArgs...>>(t)
  );
}

// Lvalue apply_tuple()...
// std::forward could have been removed. It was kept
//   - to keep appearance of code close to original
//   - to show that, if kept, const needs to be used
template <
  typename Op,
  typename... OpArgs,
  typename Indices = typename make_indices<0, OpArgs...>::type,
  template <typename...> class T
>
auto apply_tuple(Op&& op, T<OpArgs...> const& t)
  -> typename std::result_of<Op(OpArgs...)>::type
{
  return apply_tuple_impl<Indices>::apply_tuple(
    std::forward<Op>(op),
    std::forward<T<OpArgs...> const>(t)
  );
}

and apply_tuple_impl<Indices>::apply_tuple() also needs to be similarly overloaded:

template <
  typename Indices
>
struct apply_tuple_impl;

template <
  template <std::size_t...> class I,
  std::size_t... Indices
>
struct apply_tuple_impl<I<Indices...>>
{
  // Rvalue apply_tuple()...
  template <
    typename Op,
    typename... OpArgs,
    template <typename...> class T
  >
  static auto apply_tuple(Op&& op, T<OpArgs...>&& t)
    -> typename std::result_of<Op(OpArgs...)>::type
  {
    return op(
      std::forward<OpArgs>(std::get<Indices>(t))...
    );
  }

  // Lvalue apply_tuple()...
  // Notice the added const used with std::forward's OpArgs.
  template <
    typename Op,
    typename... OpArgs,
    template <typename...> class T
  >
  static auto apply_tuple(Op&& op, T<OpArgs...> const& t)
    -> typename std::result_of<Op(OpArgs...)>::type
  {
    return op(
      std::forward<OpArgs const>(std::get<Indices>(t))...
    );
  }
};

Thus, by adding overloads for constant references (i.e., lvalues) apply_tuple() can now be used with lvalues. Adding these lvalues overloads is necessary since we could not exploit the special template argument rvalue deduction rule.


Closing Comments

The special template argument rvalue deduction rule is very convenient but its applicability is limited. When the rule cannot be exploited, one needs to write overloads to handle both lvalues and rvalues. With apply_tuple() one needs to extract the OpArgs... non-type parameter which unfortunately seems to prevent one from exploiting the rule.

4 Replies to “Tweaking Applying std::tuple To Functors Efficiently”

  1. Thanks for the follow up! Overloading apply_tuple was also my first try to resolve the problem, though I hoped for a solution that avoids redundant code.

    Is there a reason why you chose to overload both apply_tuple and the method in the structure apply_tuple_impl? The forward call in apply_tuple already lifts it to an rvalue reference, so if you just use const&& in the original apply_tuple (and impl) this should avoid the need to overload apply_tuple in apply_tuple_impl (this version does work for me).

    Or does this interfere with move semantics if actually using an rvalue? At the moment I can't think of a problem there, but I'm still trying to get my head around this new concept 😉

    1. Without doing any additional metaprogramming, the overloads are needed. If you do the variations of lvalues (const, non-const) and test rvalues, you'll see that both overloads are required. (If not, then we need to exchange test cases and compare! We should also say which compilers we are using too as this is all new and perhaps not 100% in each compiler.)

      Concerning writing "const&&" I did consider what such would mean but unless it was clear what such should mean I wouldn't use it. If something is constant it should be read-only (i.e., unmovable by definition). So, unless the item is logically constant and physically not constant (e.g., due to the use of mutable members), any const rvalues should be treated as constant lvalues (e.g., "const&") for safety. Since one does not know anything about the types passed in (e.g., whether or not they are intended to be physically or logically constant), I erred on the side of safety by overloading both.

      Additionally, if one uses the presented code with suitable cases (non-const lvalues, const lvalues, and rvalues) one will need to write the overloads to overcome compiler errors: it is not sufficient to only use "const&&" instead. (Well, at least with g++ 4.7.0 snapshot 20111112 –unless I am overlooking something that's perhaps obvious?!!)

      FYI, I did want to avoid having to overload both. So far, I am 50% successful: I eliminated the need to overload the global apply_tuple. I did this by generating the list of indices without using the list of types (directly). I will be able to eliminate the extra apply_tuple inside the apply_tuple_impl struct as well. The latter is trickier however and appears to require some additional template metaprogramming. I need to get some other things done first, but, once I complete it, I may blog the details about that solution as well. However, most C++ programmers would just overload the function by copying-and-pasting it: it is simpler and doesn't need any metaprogramming –so this is what I blogged as a solution for now to get lvalues to work. (The key issue to get around is to be able to extract the OpArgs parameter pack indirectly and in a manner that works with and without qualifiers such as const. This allows one to then have Tuple&& as the apply_tuple argument and therefore elide the need to overload the static function.)

      I should say preliminarily that the solution with metaprogramming could well turn out to be simpler (i.e., there are no overloads needed), shorter (excluding the metaprogramming code –which is straightforward– as that can be tucked away in an #include library file) and certainly better for maintenance than the overloading solution.

      OTOH I could be overlooking something obvious too! That's what is nice about these blog discussions, etc. on the Internet! 🙂

    2. Thanks for your lengthy explanation, and yes I also needed both overloads! I played around a bit with the const&& and even though it might make sense to have a const rvalue reference sometimes, if you never reuse, or want to move the value you could use a const&&, but then you also could have used const& in the first place. I'm now using the overloading solution, but for what I plan to implement at the moment I might have to overload a lot of functions that basically perform the same thing, so if you have some nice template metaprogramming solution I would be really interested to see it some time.

    3. You're welcome! I may have time soon to finish a solution without the extra overloads.

      Overall, I'd would try writing code using functors (i.e., function objects) instead of function pointers. (Use std::function and std::bind when needed too.) A functor is a struct/class that overloads the function call operator. (The function call operator can also be a template member function –an example appears below.) If this is done then there are two advantages: (i) it is far more likely that the compiler will produce better code with a functor than a function pointer and (ii) full type matching and deduction will work with all arguments –unlike when you pass function pointers (be it a regular or member function).

      If you must use a function pointer and want your arguments to be deduced, then know that you can pass the type of the function pointer to apply()/apply_tuple() like this:

      template <int i> struct S { };
      template <int i> void bar(S<i> s) { }
      // …
      S<4> s;
      // Aside: writing bar(s); works as it should.
      // apply(bar,s); should ideally work but it does not unless we specify the type of the function…
      apply<decltype(bar(s))(*)(decltype(s))>(bar, s); // V1
      apply<void (*)(S<4>)>(bar, s); // V2

      where V1 is general in that you don't have to know/remember what the variable types are –however, if the number of arguments is high it is tedious to write unless you are in a template function expanding a parameter pack; and V2 is hard-coded to the types. (A typedef could also have been used.) Contrast this with:

      template <int i> struct S { };
      struct bar2 {
      template <int i>
      void operator ()(S<i> s) const { }
      };
      // …
      S<4> s;
      bar2()(s); // directly invoked
      apply(bar2(), s); // this works too!

      which is nice, clean and clearly shows argument type deduction/matching takes place in this instance.

      🙂

Leave a Reply

Your email address will not be published. Required fields are marked *