Article Discussion
Enforcing Code Feature Requirements in C++
Summary: Functions often depend on particular behavioral characteristics (“features”) of code they invoke. For example, thread-safe code must invoke only thread-safe code if it is to remain thread-safe, and exception-safe code must invoke only exception-safe code. This paper describes a technique that enables the specification of arbitrary combinations of user-defined code features on a per-function basis and that detects violations of feature constraints during compilation. The technique applies to member functions (both nonvirtual and virtual), non-member functions, and function templates; operators are excluded.
16 posts.
The ability to add new comments in this discussion is temporarily disabled.
Most recent reply: July 20, 2012 5:50 AM by Péter
    Bill
     
    Posts: 409 / Nickname: bv / Registered: January 17, 2002 4:28 PM
    Enforcing Code Feature Requirements in C++
    September 22, 2008 9:00 PM      
    Functions often depend on particular behavioral characteristics (“features”) of code they invoke. For example, thread-safe code must invoke only thread-safe code if it is to remain thread-safe, and exception-safe code must invoke only exception-safe code.
    In this article, Scott Meyers describes a technique that enables the specification of arbitrary combinations of user-defined code features on a per-function basis and that detects violations of feature constraints during compilation.
    The technique applies to member functions (both nonvirtual and virtual), non-member functions, and function templates; operators are excluded.

    Read this article here:

    http://www.artima.com/cppsource/codefeatures.html

    What do you think of the technique presented in the article?
    • dark
       
      Posts: 5 / Nickname: darkmx0z / Registered: March 11, 2008 11:38 AM
      Re: Enforcing Code Feature Requirements in C++
      September 23, 2008 8:20 AM      
      interesting article

      concepts (likely to be included in c++0x) could solve it in a more elegant way but as far as I know they can't help with standalone functions, only with types

      concept ThreadSafe<typename T> { }
      concept ExceptionSafe<typename T> { }
      concept Portable { };


      struct a {
      template<ThreadSafe F> // a will only allow to call other thread safe calls
      static void call( )
      {
      F::call( );
      }
      };

      concept_map ThreadSafe<a> { } // a is thread safe


      struct b {
      template<ExceptionSafe F> // b will only allow to call other exception safe calls
      static void call( )
      {
      F::call( );
      }
      };

      concept_map ExceptionSafe<b> { } // b is exception safe


      struct c {
      template<ThreadSafe F> requires ExceptionSafe<F> // c will require both conditions
      static void call( )
      {
      F::call( );
      }
      };

      concept_map ThreadSafe<c> { } // c is both
      concept_map ExceptionSafe<c> { }


      int main( )
      {
      a::call<b>( ); // compile-time error, b is not thread safe
      a::call<c>( ); // ok, c is thread safe

      c::call<a>( ); // compile-time error, a is not exception safe
      c::call<b>( ); // compile-time error, b is not thread safe
      c::call<c>( ); // ok, c is both thread safe and exception safe
      }
    • Hoang Vu
       
      Posts: 1 / Nickname: vu64 / Registered: April 25, 2008 1:13 AM
      Re: Enforcing Code Feature Requirements in C++
      September 23, 2008 5:54 PM      
      Glad to see Scott back on The C++ Source after two years. The topic concerned is interesting. I also want to hear more about the relationship between this and concepts in C++0x. Hope to see C++0x in your next book.
    • Krishna
       
      Posts: 1 / Nickname: krishnak / Registered: September 24, 2008 6:47 AM
      Re: Enforcing Code Feature Requirements in C++
      September 24, 2008 0:00 PM      
      Adding a remove function to the Features class as below makes explicitly removing feature when needed a bit more readable like so:


      void f(MakeFeatures<EPSafe>::type features)
      {
      }

      void g(MakeFeatures<EPSafe>::type features)
      {
      f(features);
      }

      void g(MakeFeatures<TEPSafe>::type features)
      {
      f(features.remove<ThreadSafe>());
      }



      template<typename T>
      Features<typename mpl::copy_if<S,mpl::not_<boost::is_same<_1,T> > >::type>
      remove()
      {
      return Features<typename mpl::copy_if<S,mpl::not_<boost::is_same<_1,T> > >::type>();
      }
    • Athanasios
       
      Posts: 1 / Nickname: thaaanos / Registered: September 24, 2008 11:42 PM
      Re: Enforcing Code Feature Requirements in C++
      September 25, 2008 4:50 AM      
      Well I always thought that C++ should have user defined qualifications.
      Frankly we have a complete mechanism for only 2 qualifiers : const or volatile, It's a waste.
      Why not add more or even let the user specify it's own...
      I always wanted an obsoleteor deprecated qualifier...
      I bet most c++ devealopers would have their own wishes. Maybe the list isn't that big, maybe in the next standart...
    • Rupert
       
      Posts: 3 / Nickname: rkit / Registered: December 9, 2005 7:32 AM
      Re: Enforcing Code Feature Requirements in C++
      September 24, 2008 11:04 AM      
      Interesting metaprogramming, but I am not convinced that this approach would really improve software quality for real-world projects.

      Everything is still based on programmer discipline. How does a "threadsafe" tag save me from calling strerror() instead of strerror_t()?

      Also, properties like "threadsafe" or "portable" are very hard to define and will probably change their meaning over time. This looks like a maintainance-nightmare to me.
      • Scott
         
        Posts: 5 / Nickname: sdm / Registered: August 17, 2006 10:26 AM
        Re: Enforcing Code Feature Requirements in C++
        September 24, 2008 1:38 PM      
        > Interesting metaprogramming, but I am not convinced that
        > this approach would really improve software quality for
        > real-world projects.

        I probably should have mentioned in the article that this problem was posed to me by a client. They were developing games running across the internet, and they had the notion of "deterministic" code, which, as best I can recall, was code that yielded exactly the same results, regardless of the platform on which it was run. Some of their functions were "deterministic" and some were not, and they asked me if I could think of a way to make it impossible for "deterministic" functions to call ones that were not "deterministic."

        I think that if we had tools to enforce code features, we'd find all kinds of places where they were useful.

        > Everything is still based on programmer discipline. How
        > does a "threadsafe" tag save me from calling strerror()
        > instead of strerror_t()?

        Enforcing code features doesn't save you from all your errors, but it gives you a tool to enforce certain interface contracts that are currently invisible to compilers. Think of it as a new feature of the type system. If you find type systems useful, I'd hope you'd see that this new feature could help prevent certain kinds of errors.

        > Also, properties like "threadsafe" or "portable" are very
        > hard to define and will probably change their meaning over
        > time. This looks like a maintainance-nightmare to me.

        Current code already depends on code features, and the definitions of those features are already subject to change. If the definition of some feature (such as "thread safe") changes, functions that depend on the new definition will likely behave improperly when calling functions offering only the old definition. We currently catch such errors via code reviews, testing, and debugging. Using the approach I describe, we'd just change the name of the code feature (e.g., from "ThreadSafeV1" to "ThreadSafeV2", possibly having the latter inherit from the former, as suggested in the "Open Issues" section of the article), update the code to use the new feature names in the appropriate places, and recompile. If the result was not consistent, the compiler would complain. I don't see how this would be any worse than the way we currently do things, and I'd expect it to be better.

        Scott
        • Rupert
           
          Posts: 3 / Nickname: rkit / Registered: December 9, 2005 7:32 AM
          Re: Enforcing Code Feature Requirements in C++
          September 25, 2008 0:07 PM      
          > I think that if we had tools to enforce code features,
          > we'd find all kinds of places where they were useful.
          >

          I agree, but I think those tools should enforce these features by verifying the sematics of the code, not just checking labels attached to functions.

          But where I have to rely on expert knowledege and dicipline, I really prefer assigning attributes to much larger code entities, like complete libraries, or at least at class level. When you start to track something like concurrency at the function level, chances are you are already in big trouble.
          • dark
             
            Posts: 5 / Nickname: darkmx0z / Registered: March 11, 2008 11:38 AM
            Re: Enforcing Code Feature Requirements in C++
            September 25, 2008 1:03 PM      
            Since the const qualification to member functions has worked very well, its likely that a generalization of that semantic checking would prove useful.

            In fact I find that what Athanasios Kakargias is proposing is trying to be emulated by Scott without language support.

            Given that C++0x will have native multithreading support, Scott's and Athanasios' ideas could give us a very elegant solution.
          • Scott
             
            Posts: 5 / Nickname: sdm / Registered: August 17, 2006 10:26 AM
            Re: Enforcing Code Feature Requirements in C++
            September 25, 2008 1:37 PM      
            > I agree, but I think those tools should enforce these
            > features by verifying the sematics of the code, not just
            > checking labels attached to functions.

            Of course, but this is tough to do for arbitrary characteristics. Look at how much work went into Concepts in C++0x, yet it doesn't even try to verify semantics. As far as I know, no commercially important programming language has any support for compiler checking of user-specified semantics that's any more sophisticated than verifying that a user has asserted that a piece of code has the semantics desired (which is essentially what I'm trying to offer). Note that even "const" in C++ doesn't really respect program semantics, because the compiler enforces bitwise constness, but semantically we want logical constness, and the two are not always the same.

            > But where I have to rely on expert knowledege and
            > dicipline, I really prefer assigning attributes to much
            > larger code entities, like complete libraries, or at least
            > at class level. When you start to track something like
            > concurrency at the function level, chances are you are
            > already in big trouble.

            I'm sure the proper granularity depends on the feature. Currently, for example, all exception safety analysis is done on a per-function basis. Similarly, the proper granularity for "side-effect-freeness" is probably the function.

            Scott
    • Rich
       
      Posts: 2 / Nickname: rdsposato / Registered: March 15, 2006 7:18 PM
      Re: Enforcing Code Feature Requirements in C++
      October 13, 2008 2:10 PM      
      This is a really neat ability! My hat's off to Scott Meyers for coming up with a mechanism to check any number of arbitrary constraints on a function at compile time.

      I have a few complaints about it. As I discuss this trick, I'd like to use the word "constraints" rather than "features". I know that seems pedantic, but I see Scott Meyer's technique as all about enforcing constraints on functions. Also, I would use the word consistent to describe code that runs the same on multiple platforms - rather than deterministic.

      1. What if I want to call some code I found in an open source library or provided by a vendor? I may have no guarantees the code is exception safe, thread safe, or consistent. If use Meyer's code-constraint feature within my functions, but still call other code, I can no longer claim mine is exception safe, thread safe, or consistent. This trick only works if I can pass a parameter into a function and I can't do that with every function I call.

      2. It requires passing the features into functions as parameters. I see this more as a weakness akin to Achille's Heal as described in the above paragraph. Is there any to enforce this without using parameters as the mechanism? Something less intrusive?

      3. Let's say I already write code and use this trick to show it is exception safe, thread safe, and consistent - and then release my code to users. What if I later want to add more constraints (e.g. - now I say it is portable and has a Big-O constraint of n*n operations). Once my code is out in the wild, I don't see how I can add more constraints to it.

      4. As mentioned in a comment by Rupert Kittinger, definitions of thread-safe and portable will vary over time. (e.g. - Today I may call some code portable if it compiles without errors on compilers A and B. Next week, I may decide it is portable if it compiles without errors on compilers B, C, and D.)

      Despite these issues, I think Scott Meyer's technique has promise. Again, I compliment you on creating this.
    • Jesse
       
      Posts: 1 / Nickname: chardan / Registered: December 28, 2005 1:20 PM
      Re: Enforcing Code Feature Requirements in C++
      January 21, 2010 11:39 PM      
      Hi, I have a question. Is it fair to consider this approach as a case of programming with dependent types? Put another way, is this use acceptable to be thought of as an example of using weakly-dependent types in C++?

      _Jesse Williamson ;-};
    • viboes
       
      Posts: 1 / Nickname: viboes / Registered: February 3, 2005 8:25 AM
      Re: Enforcing Code Feature Requirements in C++
      March 30, 2010 9:52 PM      
      Really interesting. Thanks to all the people that have contribute to this.

      The intrusive use of the features parameter could be mitigated if we were able to state the features of a function as function traits.


      template <typename FUNCTION>
      struct features;

      struct features<void f(int, double)> {
      typedef make_features<ThreadSafe, ExceptionSafe>::type type;
      };

      struct features<void g()> {
      typedef make_features<ThreadSafe, ExceptionSafe, Portable>::type type;
      };



      We can define a metafunction that states if a given function can call another one


      template <typename From, typename To>
      struct can_call {
      ...
      };


      Now the user needs just to prefix each call with a static assertion of the calls the function made


      void f(int x, double y) {
      ...
      static_assert(can_call<void f(int, double), void g()>::value>;
      g();
      ...
      }


      If we want to avoid the separation of the assertion and the call we can define a forward macro that will statically check if the function can be called.


      #define CALL_PROC_FROM(FROM, TO) \
      if (false) { static_assert(can_call<FROM, typeof(TO)>::value);}
      else TO


      void f(int x, double y) {
      ...
      CALL_PROC_FROM(void f(int, double), g());
      ...
      }


      we can yet define a macro that uses a CURRENT_FUNCTION macro to add some syntax sugar


      #define CALL_PROC(TO) CALL_PROC_FROM(CURRENT_FUNCTION, TO)

      void f(int x, double y) {
      #define CURRENT_FUNCTION void f(int, double)
      ...
      CALL_PROC(g());
      ...
      #undef CURRENT_FUNCTION
      }



      We can define also features associated to a class


      struct features<C> {
      typedef make_features<ThreadSafe>::type type;
      };


      implying that all the functions of this class share this features, that is the features of the function are the features of the class plus the explicit features of the function.

      the metafunction 'can_call' will take care of this.


      Advantage of this approach:
      * Support controlled call to 3pp functions.
      * Support operators.
      * Allows to specify code features for groups of functions.
      * No impact on runtime performance.
      * Improve diagnostics: static assert is enough explicit.
      * the user can specialize the can_call metafunction for specific functions directly.

      Liabilities:
      * Unable to call different functions depending on the provided features. This needs to maintain the feature parameter.

      What this approach doesn't address:
      * Supporting inheritance among feature classes
      * Elimination of AllCodeFeatures
      * Improving compile-time performance
      * Allowing users to control feature set type conversion policy (see liabilities)
      • Stanislav
         
        Posts: 1 / Nickname: pyatykh / Registered: May 11, 2012 3:20 AM
        Re: Enforcing Code Feature Requirements in C++
        May 11, 2012 11:32 AM      
        The last article in The C++ Source was published in 2008.
        Is the journal closed?
        • Bill
           
          Posts: 409 / Nickname: bv / Registered: January 17, 2002 4:28 PM
          Re: Enforcing Code Feature Requirements in C++
          May 12, 2012 6:59 AM      
          > The last article in The C++ Source was published in 2008.
          > Is the journal closed?
          >
          Yes, it essentially closed down once the founding editor left.

          Bill
    • Péter
       
      Posts: 2 / Nickname: petersohn / Registered: July 4, 2012 9:54 PM
      Re: Enforcing Code Feature Requirements in C++
      July 5, 2012 3:01 AM      
      Interesting concept, but very inefficient. I really could not get over the exponential virtual inharitance tree. I was thinking whether there is a better version, and came up with one that uses implicit conversion rules instead of virtual inheritance. Here is the source code:

      #ifndef CODEFEATURES_HPP_
      #define CODEFEATURES_HPP_

      #include <boost/mpl/vector.hpp>
      #include <boost/mpl/placeholders.hpp>
      #include <boost/mpl/find.hpp>
      #include <boost/mpl/less.hpp>
      #include <boost/mpl/copy.hpp>
      #include <boost/mpl/fold.hpp>
      #include <boost/mpl/sort.hpp>
      #include <boost/mpl/remove_if.hpp>
      #include <boost/mpl/empty_base.hpp>
      #include <boost/mpl/transform.hpp>
      #include <boost/mpl/contains.hpp>
      #include <boost/mpl/and.hpp>
      #include <boost/utility/enable_if.hpp>


      namespace CodeFeatures {

      using boost::mpl::_1;
      using boost::mpl::_2;
      using boost::mpl::_;

      template<typename S>
      struct Features
      {
      typedef S features_type;

      template <typename OtherFeature>
      Features(
      const OtherFeature&,
      typename boost::enable_if<
      typename boost::mpl::fold<
      typename OtherFeature::features_type,
      boost::mpl::true_,
      boost::mpl::and_<
      boost::mpl::contains<S, _2>,
      _1
      >
      >::type,
      int
      >::type = 0)
      {}
      };

      }

      #endif /* CODEFEATURES_HPP_ */


      As it can be seen, it eliminates the need of AllCodeFeatures as well, so it can be generically applied to any feature set. It also works for larger amount of features. For example:


      namespace mpl = boost::mpl;
      namespace cf = CodeFeatures;

      struct F1{};
      struct F2{};
      struct F3{};
      struct F4{};
      struct F5{};
      struct F6{};
      struct F7{};
      struct F8{};
      struct F9{};
      struct F10{};
      struct F11{};
      struct F12{};

      BOOST_STATIC_ASSERT((
      boost::is_convertible<
      cf::Features<mpl::vector<F4,F5,F6,F7,F8> >,
      cf::Features<mpl::vector<F1,F2,F3,F4,F5,F6,F7,F8,F9,F10,F11,F12> >
      >::value));

      BOOST_STATIC_ASSERT((
      boost::is_convertible<
      cf::Features<mpl::vector<F1,F2,F3,F4,F5,F6,F7,F8,F9,F10,F11,F12> >,
      cf::Features<mpl::vector<F1,F2,F3,F4,F5,F6,F7,F8,F9,F10,F11,F12> >
      >::value));

      BOOST_STATIC_ASSERT((
      !boost::is_convertible<
      cf::Features<mpl::vector<F1,F2,F3,F4,F5,F6,F7,F8,F9,F10,F11,F12> >,
      cf::Features<mpl::vector<F1,F2,F3,F4,F5,F6,F8,F9,F10,F11,F12> >
      >::value));
      • Péter
         
        Posts: 2 / Nickname: petersohn / Registered: July 4, 2012 9:54 PM
        Re: Enforcing Code Feature Requirements in C++
        July 20, 2012 5:50 AM      
        I was thinking further about my idea, and realized the problem with this. It works well for basic purposes: to enforce certain functions to have a set of features. However, it doesn't work for automatic overload resolution. For example, if we have a function with an overload that is thread safe, and another that is both thread safe and exception safe, and we call it with a requirement of being only thread safe, we will get an error of ambiguous overload.

        The original solution uses virtual inheritance, which provides a built-in overload resolution choosing the closest possible match. With implicit conversion, we can't use this algorithm. We have to choose the overload "manually" via enable_if. I built a forwarding of enable_if that computes this without the need to write large expressions for every overload. The disadvantage is that we need to know all the overloads in advance. Here is how it works.


        namespace mpl = boost::mpl;
        namespace cf = CodeFeatures;

        typedef cf::Features<mpl::vector<ThreadSafe>> TSafe;
        typedef cf::Features<mpl::vector<ThreadSafe, ExceptionSafe>> TESafe;

        typedef mpl::vector<TSafe, TESafe> CalleeFeaturesList;

        template <class Features>
        typename cf::SelectFeatures<Features, TSafe, CalleeFeaturesList>::type
        callee()
        {
        std::cout << "TSafe" << std::endl;
        }


        template <class Features>
        typename cf::SelectFeatures<Features, TESafe, CalleeFeaturesList>::type
        callee()
        {
        std::cout << "TESafe" << std::endl;
        }


        This version has advantages over the original implementation as well. First, as it uses templates and decides which overload to use with the metafunction on the top, we don't even need to instantiate the feature variables. Second, the template makes these functions pass its requirement transitively. For example, we have a caller function like this:


        template <class Features>
        typename cf::SelectFeatures<Features, TESafe, mpl::vector<TESafe>>::type
        caller()
        {
        callee<Features>();
        }


        This function provides both thread safety and exception safety to its callers, and has no overloads. However, if we only need thread safety, it will call the TSafe version of callee, providing a more efficient program without needing to use overloads over the whole call hierarchy.

        Of course, if we don't want to use templates, the original approach works just as fine:


        typedef TESafe callerFeatures;
        void callerNoTemplate(callerFeatures features)
        {
        calleeNoTemplate(features); // the original way
        callee<callerFeatures>(); // of course we can't use the variable in the template parameter. We have to use the type name itself.
        }




        Here is the implementation. First, I introduced some new metafunctions because the lack of contains_if in Boost MPL.


        template <typename Seq, typename Pred>
        struct none_of: boost::is_same<
        typename boost::mpl::find_if<Seq, Pred>::type,
        typename boost::mpl::end<Seq>::type
        > {};

        template <typename Seq, typename Pred>
        struct all_of: none_of<
        Seq,
        boost::mpl::not_<Pred>
        > {};


        With this, even the definition of Features becomes easier. Otherwise, it does the same as before (except that in my last post, I left out the default constructor).


        template <typename S>
        struct Features
        {
        typedef S features_type;

        Features() {}

        template <typename OtherFeature>
        Features(
        const OtherFeature&,
        typename boost::enable_if<
        typename detail::all_of<
        typename OtherFeature::features_type,
        boost::mpl::contains<S, _>
        >::type,
        int
        >::type = 0)
        {}
        };


        Now comes the implementation for SelectFeatures. In human words, it does the following: "Enable this overload if T is convertible to ThisType and not convertible to any types in AllTypes which ThisType is also not convertible to."


        template <typename T, typename ThisType, typename AllTypes, typename ValueType = void>
        struct SelectFeatures: boost::enable_if<
        boost::mpl::and_<
        boost::is_convertible<T, ThisType>,
        detail::all_of<
        AllTypes,
        boost::mpl::or_<
        boost::is_convertible<ThisType, _1>,
        boost::mpl::not_<
        boost::is_convertible<T, _1>
        >
        >
        >
        >,
        ValueType
        > {};