How to retrieve embedded collections?

[michalbundyra]

Not sure, which place is more suitable for this question -- here or the ZF HAL issues page...

I'm working at an Apigility driven application with following end points:

/projects
/projects/:id
/images
/images/:id

The relationship between Project and Image is 1:n.

The configs render_embedded_entities and render_collections (see Apigility documentation: "ZF HAL -> Configuration -> User Configuration -> renderer") are not set, that means set to the default value (true). Somehow it didn't work for me, so I implemented this in the ProjectService class:

...

class ProjectService implements ServiceManagerAwareInterface {

    ...

    public function getProject($id) {
        $project = $this->getMapper()->findById($id);
        $images = $this->getImageService()->getImagesForProject($id);
        $project->setImages($images);
        return $project;
    }

    ...

}

The /projects/:id end point provides a project with an image list. Well. Now I want to achieve the same behavior for the /projects -- every element should provide a list of images, that belong to it. The implementation like

public function getProjects() {
    $projects = $this->getMapper()->findAll();
    foreach ($projects as $project) {
        $images = $this->getImageService()->getImagesForProject($project->getId());
        $project->setImages($images);
    }
    return $projects;
}

is not possible, since a collection is not a list of single entity elements.

So how to implement this?

---

Originally posted by @automatix at zfcampus/zf-apigility#90

[michalbundyra]

I'm having the exact same problem. Did you ever solve this?

---

Originally posted by @poisa at zfcampus/zf-apigility#90 (comment)

[michalbundyra]

Unfortunately not. I tried different ways to solve it and "spammed" stackoverflow with thousand Apigility related questions (nested responses, linking collections to theentities, Doctrine, data manipulation before sending to the client, and some other), but I have not found neither a solution, nor even an acceptable workaround.

I wonder, why this basic question cannot find an answer. Nearly every application, that is more complex than "Hello World!", needs nested output. I'm sure, it must be possible with Apigility, and would be glad to get some information, how to do this. And it would be nice, if the Documentation would be extended with an example with a nested ouput (like a projects list { ... _embedded: { projects: [ { ... _embedded: { images: [ { image }, { image }, { image } ] } }, { ... _embedded: { images: [ { image }, { image }, { image } ] } }, ... ] } }).

---

Originally posted by @automatix at zfcampus/zf-apigility#90 (comment)

[michalbundyra]

Hey, I think I figured this out. At least, I found a way to do this, though I don't know if it is the correct way. I saw a post by Matthew on the Google groups addressing a similar issue and changed one line to make it work for embedding other resources. I've left the FQCN in to be clearer.
In my case I have a Member entity which has a Membership embedded entity.

// MembersResource.php
    /**
     * Fetch all or a subset of resources
     *
     * @param  array $params
     * @return ApiProblem|mixed
     */
    public function fetchAll($params = array())
    {
        $resultSet = new \Zend\Db\ResultSet\HydratingResultSet(
            $this->services->get('Application\Model\Hydrator\MemberHydrator'),
            new \Application\Model\Entity\Member
        );

        $select = new \Zend\Db\Sql\Select('members');
        $paginatorAdapter = new \Zend\Paginator\Adapter\DbSelect($select, $this->services->get('Zend\Db\Adapter\Adapter'), $resultSet);
        $collection = new \Admin\V1\Rest\Members\MembersCollection($paginatorAdapter);
        return $collection;
    }

The key here is the custom MemberHydrator I'm passing in. It's almost an empty class that extends Zend\Stdlib\Hydrator\ClassMethods but overrides the extract() and hydrate() methods.
I'm still working on this implementation (haven't even finished my extract() method properly, but the hydrate() method goes something like this:

//MemberHydrator
    /**
     * Hydrate an object by populating getter/setter methods
     *
     * Hydrates an object by getter/setter methods of the object.
     *
     * @param  array $data
     * @param  object $object
     * @return object
     * @throws Exception\BadMethodCallException for a non-object $object
     */
    public function hydrate(array $data, $object)
    {
        $object = parent::hydrate($data, $object);

        if ($object->getMembershipId() !== null)
        {
            $membership = $this->membershipMapper->findById($object->getMembershipId());
        }
        $object->setMembershipId($membership);
        return $object;
    }

Basically when hydrating, if the getMembershipId() method is not null (in my case its an INT column) then I find that record on the membership table and replace the id on my member table with the membership entity object. (Note, I will be renaming membershipId with membership). If you have more embedded entities, this is the place to embed them.

Now, my fetchAll() returns a proper collection with working pagination and embedded entities.

Pros: Easy to code.
Cons: Makes an extra db query per embedded entity.

I'm going to attempt a version 2 of this where the Select object does a JOIN with the related table and brings in all the necessary fields so that the MemberHydrator does not have to make any extra db calls.

I completely agree with you that this should be documented somewhere as I too think this is a very basic need. If Matthew ever confirms that this is the way to go, I will gladly submit a PR to the docs repo with a recipe to do this.

Caveats: (and I think this too should be part of Apigility at least in an optional manner) If you PUT/POST/PATCH a document with embedded resources back to Apigility it will complain about the unknown _embedded and _links fields and will stop dead on its tracks. It would be very nice to have a way to declare your embedded resources so that Apigility knows what to expects.

---

Originally posted by @poisa at zfcampus/zf-apigility#90 (comment)

[michalbundyra]

Trying to implement you solution... How do you make the Mapper class available in the custom Hydrator? Are you passing the ServiceManager into it somehow? Can you please post the whole class code here or a link to it?

---

Originally posted by @automatix at zfcampus/zf-apigility#90 (comment)

[michalbundyra]

Pretty much like you said: I inject the ServiceManager into the Hydrator since I will need to pull out many mappers, none of which I know the name yet. Supposedly composing the ServiceManager is not a good practice but in this case I think I have a good case for it.

Note that in my case I am subclassing the ClassMethods hydrator since it does 99% of the work for me. I'm including an entity so that you know what they look like. This is an early version with just one embedded entity (which suffices for an example). Note that I autogenerate my entities from a little script that I wrote which introspects the MySQL tables so that I don't have to code everything again with every schema change. This means that if I name my database fields correctly, the rest is done for me: I can use just ONE hydrator and it will work with every entity.

<?php
// EmbeddedEntity.php
namespace Application\Model\Hydrator;

use Zend\Stdlib\Exception;
use Zend\Stdlib\Hydrator\ClassMethods as ClassMethodsHydrator;

class EmbeddedEntity extends ClassMethodsHydrator
{
    /**
     * @var \Zend\ServiceManager\ServiceManager
     */
    protected $serviceManager;

    public function __construct()
    {
        parent::__construct(false);
    }

    /**
     * Extract values from an object with class methods
     *
     * Extracts the getter/setter of the given $object.
     *
     * @param  object $object
     * @return array
     * @throws Exception\BadMethodCallException for a non-object $object
     */
    public function extract($object)
    {
        $data = parent::extract($object);

        foreach ($object->getSpecialFields() as $realField => $entityField)
        {
            unset($data[$entityField]);
        }
        unset($data['specialFields']);
        return $data;
    }

    /**
     * Hydrate an object by populating getter/setter methods
     *
     * Hydrates an object by getter/setter methods of the object.
     *
     * @param  array $data
     * @param  object $object
     * @return object
     * @throws Exception\BadMethodCallException for a non-object $object
     */
    public function hydrate(array $data, $object)
    {
        $object = parent::hydrate($data, $object);

        foreach ($object->getSpecialFields() as $realField => $entityField)
        {
            // $realField is "membershipId"
            // $entityField is "membership"
            // Now I find my membership entity and do something like
            // $object->membership = $membershipMapper->findById(...)
        }

        return $object;
    }

    /**
     * @return \Zend\ServiceManager\ServiceManager
     */
    public function getServiceManager()
    {
        return $this->serviceManager;
    }

    /**
     * @param \Zend\ServiceManager\ServiceManager $serviceManager
     */
    public function setServiceManager($serviceManager)
    {
        $this->serviceManager = $serviceManager;
    }
}

Here's the entity:

/// Member.php

<?php

namespace Application\Model\Entity;

class Member implements MemberInterface
{

    protected $id = '';

    protected $name = '';

    protected $email = '';

    protected $email2 = '';

    protected $validatedEmail = '';

    protected $activateCode = '';

    protected $city = '';

    protected $address = '';

    protected $phone1 = '';

    protected $phone2 = '';

    protected $phone3 = '';

    protected $isMember = '';

    protected $wantsMail = '';

    protected $birthdate = '';

    protected $creationDate = '';

    protected $memberSince = '';

    protected $membershipId = '';

    protected $membership = null; // <-- not a real field in the DB but an embedded entity

    protected $isInstitute = '';

    protected $instituteName = '';

    protected $comments = '';

    protected $plays = '';

    // This is how I tell my hydrator, what database column "goes" into what
    // embedded entity property. In this case, I'm saying "put a Membership entity
    // into the membership property and find it by membershipId in the membership table"
    protected $specialFields = array(
        'membershipId' => 'membership'
    );

    /**
     * @return int
     */
    public function getId()
    {
        return $this->id;
    }

    /**
     * @param int $id
     * @return \Application\Model\Entity\Members
     */
    public function setId($id)
    {
        $this->id = $id;
        return $this;
    }

    /**
     * @return string
     */
    public function getName()
    {
        return $this->name;
    }

    /**
     * @param string $name
     * @return \Application\Model\Entity\Members
     */
    public function setName($name)
    {
        $this->name = $name;
        return $this;
    }

    /**
     * @return string
     */
    public function getEmail()
    {
        return $this->email;
    }

    /**
     * @param string $email
     * @return \Application\Model\Entity\Members
     */
    public function setEmail($email)
    {
        $this->email = $email;
        return $this;
    }

    /**
     * @return string
     */
    public function getEmail2()
    {
        return $this->email2;
    }

    /**
     * @param string $email2
     * @return \Application\Model\Entity\Members
     */
    public function setEmail2($email2)
    {
        $this->email2 = $email2;
        return $this;
    }

    /**
     * @return int
     */
    public function getValidatedEmail()
    {
        return $this->validatedEmail;
    }

    /**
     * @param int $validatedEmail
     * @return \Application\Model\Entity\Members
     */
    public function setValidatedEmail($validatedEmail)
    {
        $this->validatedEmail = $validatedEmail;
        return $this;
    }

    /**
     * @return string
     */
    public function getActivateCode()
    {
        return $this->activateCode;
    }

    /**
     * @param string $activateCode
     * @return \Application\Model\Entity\Members
     */
    public function setActivateCode($activateCode)
    {
        $this->activateCode = $activateCode;
        return $this;
    }

    /**
     * @return string
     */
    public function getCity()
    {
        return $this->city;
    }

    /**
     * @param string $city
     * @return \Application\Model\Entity\Members
     */
    public function setCity($city)
    {
        $this->city = $city;
        return $this;
    }

    /**
     * @return string
     */
    public function getAddress()
    {
        return $this->address;
    }

    /**
     * @param string $address
     * @return \Application\Model\Entity\Members
     */
    public function setAddress($address)
    {
        $this->address = $address;
        return $this;
    }

    /**
     * @return string
     */
    public function getPhone1()
    {
        return $this->phone1;
    }

    /**
     * @param string $phone1
     * @return \Application\Model\Entity\Members
     */
    public function setPhone1($phone1)
    {
        $this->phone1 = $phone1;
        return $this;
    }

    /**
     * @return string
     */
    public function getPhone2()
    {
        return $this->phone2;
    }

    /**
     * @param string $phone2
     * @return \Application\Model\Entity\Members
     */
    public function setPhone2($phone2)
    {
        $this->phone2 = $phone2;
        return $this;
    }

    /**
     * @return string
     */
    public function getPhone3()
    {
        return $this->phone3;
    }

    /**
     * @param string $phone3
     * @return \Application\Model\Entity\Members
     */
    public function setPhone3($phone3)
    {
        $this->phone3 = $phone3;
        return $this;
    }

    /**
     * @return int
     */
    public function getIsMember()
    {
        return $this->isMember;
    }

    /**
     * @param int $isMember
     * @return \Application\Model\Entity\Members
     */
    public function setIsMember($isMember)
    {
        $this->isMember = $isMember;
        return $this;
    }

    /**
     * @return int
     */
    public function getWantsMail()
    {
        return $this->wantsMail;
    }

    /**
     * @param int $wantsMail
     * @return \Application\Model\Entity\Members
     */
    public function setWantsMail($wantsMail)
    {
        $this->wantsMail = $wantsMail;
        return $this;
    }

    /**
     * @return string
     */
    public function getBirthdate()
    {
        return $this->birthdate;
    }

    /**
     * @param string $birthdate
     * @return \Application\Model\Entity\Members
     */
    public function setBirthdate($birthdate)
    {
        $this->birthdate = $birthdate;
        return $this;
    }

    /**
     * @return string
     */
    public function getCreationDate()
    {
        return $this->creationDate;
    }

    /**
     * @param string $creationDate
     * @return \Application\Model\Entity\Members
     */
    public function setCreationDate($creationDate)
    {
        $this->creationDate = $creationDate;
        return $this;
    }

    /**
     * @return string
     */
    public function getMemberSince()
    {
        return $this->memberSince;
    }

    /**
     * @param string $memberSince
     * @return \Application\Model\Entity\Members
     */
    public function setMemberSince($memberSince)
    {
        $this->memberSince = $memberSince;
        return $this;
    }

    public function getMembership()
    {
        return $this->membership;
    }

    public function setMembership($membership)
    {
        $this->membership = $membership;
        return $this;
    }

    /**
     * @return int
     */
    public function getMembershipId()
    {
        return $this->membershipId;
    }

    /**
     * @param int $membershipId
     * @return \Application\Model\Entity\Members
     */
    public function setMembershipId($membershipId)
    {
        $this->membershipId = $membershipId;
        return $this;
    }

    /**
     * @return int
     */
    public function getIsInstitute()
    {
        return $this->isInstitute;
    }

    /**
     * @param int $isInstitute
     * @return \Application\Model\Entity\Members
     */
    public function setIsInstitute($isInstitute)
    {
        $this->isInstitute = $isInstitute;
        return $this;
    }

    /**
     * @return string
     */
    public function getInstituteName()
    {
        return $this->instituteName;
    }

    /**
     * @param string $instituteName
     * @return \Application\Model\Entity\Members
     */
    public function setInstituteName($instituteName)
    {
        $this->instituteName = $instituteName;
        return $this;
    }

    /**
     * @return string
     */
    public function getComments()
    {
        return $this->comments;
    }

    /**
     * @param string $comments
     * @return \Application\Model\Entity\Members
     */
    public function setComments($comments)
    {
        $this->comments = $comments;
        return $this;
    }

    /**
     * @return string
     */
    public function getPlays()
    {
        return $this->plays;
    }

    /**
     * @param string $plays
     * @return \Application\Model\Entity\Members
     */
    public function setPlays($plays)
    {
        $this->plays = $plays;
        return $this;
    }

    public function getSpecialFields()
    {
        return $this->specialFields;
    }


}

---

Originally posted by @poisa at zfcampus/zf-apigility#90 (comment)

[michalbundyra]

Thank you for your datailed example! It's not working for me yet. The problem is, that the entity hydration settings seem not to work for collection calls (SO question here). I've debugged a litttle and haven't observe any hydrate() or extract() calls on a hydrator. Whatever I set for the entity hydration, when a collection is retrieved, neither ClassMethods, nor other hydrators are called.

Module class

class Module implements ApigilityProviderInterface {
    ...
    public function getServiceConfig() {
        return array(
            'factories' => array(
                ...
                'MyNamespace\\Hydrator\\ProjectHydrator' => function(ServiceManager $serviceManager) {
                    $projectHydrator = new ProjectHydrator();
                    $projectHydrator->setImageService($serviceManager->get('Portfolio\V2\Rest\ImageService'));
                    return $projectHydrator;
                }
            ),
            ...
        );
    }
}

module.config.php

...
'zf-hal' => array(
    'metadata_map' => array(
        ...
        'Portfolio\\V2\\Rest\\Project\\ProjectEntity' => array(
            'entity_identifier_name' => 'id',
            'route_name' => 'portfolio.rest.project',
            'route_identifier_name' => 'id',
            // 'hydrator' => 'Zend\\Stdlib\\Hydrator\\ClassMethods',
            'hydrator' => 'MyNamespace\\Hydrator\\ProjectHydrator',
        ),
        ...
    ),
),
...

How have you configured the system for using your EmbeddedEntity hydrator?

---

Originally posted by @automatix at zfcampus/zf-apigility#90 (comment)

[michalbundyra]

Your config looks like mine. In my case, the only way to properly hydrate collection calls was to use what I posted in this comment.

Here's a fun fact: in my module.config.php I am using the ClassMethods hydrator. Well, actually I'm using what I call a Generic hydrator which is exactly the same but it configures the $underscoreSeparatedKeys to false:

<?php

namespace Application\Model\Hydrator;

use Zend\Stdlib\Hydrator\ClassMethods as ClassMethodsHydrator;

class Generic extends ClassMethodsHydrator
{
    public function __construct()
    {
        parent::__construct(false);
    }
}

So to all intents and purposes this is exactly the same as the ClassMethods hydrator. If I use my own hydrator here, I get flat responses from Apigility and if I use this hydrator I get the properly embedded responses. Sounds backwards but I think it has to do with me already providing Apigility with properly hydrated entities:

    /**
     * Fetch a resource
     *
     * @param  mixed $id
     * @return ApiProblem|mixed
     */
    public function fetch($id)
    {
        // My mapper is already using my own hydrator which already returns all the embedded 
        // entities where they should be
        return $this->mapper->findById($id);
    }

---

Originally posted by @poisa at zfcampus/zf-apigility#90 (comment)

[michalbundyra]

It wokrs! I want to post my variant of your (@poisa) solution:

/module/Portfolio/config/module.config.php

return array(
    ...
    'zf-hal' => array(
        'metadata_map' => array(
            ...
            'Portfolio\\V2\\Rest\\Project\\ProjectEntity' => array(
                'entity_identifier_name' => 'id',
                'route_name' => 'portfolio.rest.project',
                'route_identifier_name' => 'id',
                'hydrator' => 'Portfolio\\V2\\Rest\\Project\\ProjectHydrator',
            ),
            'Portfolio\\V2\\Rest\\Project\\ProjectCollection' => array(
                'entity_identifier_name' => 'id',
                'route_name' => 'portfolio.rest.project',
                'route_identifier_name' => 'id',
                'is_collection' => true,
            ),
            ...
        ),
    ),
);

Portfolio\Module

class Module implements ApigilityProviderInterface {

    ...

    public function getHydratorConfig() {
        return array(
            'factories' => array(
                // V2
                'Portfolio\\V2\\Rest\\Project\\ProjectHydrator' => function(ServiceManager $serviceManager) {
                    $projectHydrator = new ProjectHydrator();
                    $projectHydrator->setImageService($serviceManager->getServiceLocator()->get('Portfolio\V2\Rest\ImageService'));
                    return $projectHydrator;
                }
            ),
        );
    }

    ...

}

Portfolio\V2\Rest\Project\ProjectHydrator

namespace Portfolio\V2\Rest\Project;

use Zend\Stdlib\Hydrator\ClassMethods;
use Portfolio\V2\Rest\Image\ImageService;

class ProjectHydrator extends ClassMethods {

    /**
     * @var ImageService
     */
    protected $imageService;

    /**
     * @return ImageService the $imageService
     */
    public function getImageService() {
        return $this->imageService;
    }

    /**
     * @param ImageService $imageService
     */
    public function setImageService(ImageService $imageService) {
        $this->imageService = $imageService;
        return $this;
    }

    /*
     * Doesn't need to be implemented:
     * the ClassMethods#hydrate(...) handle the $data already as wished.
     */
    /*
    public function hydrate(array $data, $object) {
        $object = parent::hydrate($data, $object);
        if ($object->getId() !== null) {
            $images = $this->imageService->getImagesForProject($object->getId());
            $object->setImages($images);
        }
        return $object;
    }
    */

    /**
     * @see \Zend\Stdlib\Hydrator\ClassMethods::extract()
     */
    public function extract($object) {
        $array = parent::extract($object);
        if ($array['id'] !== null) {
            $images = $this->imageService->getImagesForProject($array['id']);
            $array['images'] = $images;
        }
        return $array;
    }

}

Portfolio\V2\Rest\Project\ProjectMapperFactory

namespace Portfolio\V2\Rest\Project;

use Zend\ServiceManager\ServiceLocatorInterface;

class ProjectMapperFactory {

    public function __invoke(ServiceLocatorInterface $serviceManager) {
        $mapper = new ProjectMapper();
        $mapper->setDbAdapter($serviceManager->get('PortfolioDbAdapter_V2'));
        $mapper->setEntityPrototype($serviceManager->get('Portfolio\V2\Rest\Project\ProjectEntity'));
        $projectHydrator = $serviceManager->get('HydratorManager')->get('Portfolio\\V2\\Rest\\Project\\ProjectHydrator');
        $mapper->setHydrator($projectHydrator);
        return $mapper;
    }

}

Portfolio\V2\Rest\Project\ProjectMapper

namespace Portfolio\V2\Rest\Project;

use ZfcBase\Mapper\AbstractDbMapper;
use Zend\Paginator\Adapter\DbSelect;
use Zend\Db\ResultSet\HydratingResultSet;

class ProjectMapper extends AbstractDbMapper {

    ...

    /**
     * Provides a collection of all the available projects.
     *
     * @return \Portfolio\V2\Rest\Project\ProjectCollection
     */
    public function findAll() {
        $resultSetPrototype = new HydratingResultSet(
            $this->getHydrator(),
            $this->getEntityPrototype()
        );
        $select = $this->getSelect();
        $adapter = $this->getDbAdapter();
        $paginatorAdapter = new DbSelect($select, $adapter, $resultSetPrototype);
        $collection = new ProjectCollection($paginatorAdapter);
        return $collection;
    }

    /**
     * Provides a project by ID.
     *
     * @param int $id
     * @return \Portfolio\V2\Rest\Project\ProjectEntity
     */
    public function findById($id) {
        $select = $this->getSelect();
        $select->where(array(
            'id' => $id,
        ));
        $entity = $this->select($select)->current();
        return $entity;
    }

    ...

}

It would be great to get a feedback from someone from the Apigility core team, wheter this solution is "Apigility conform" add, if not, what is a better/"correct" solution.

---

Originally posted by @automatix at zfcampus/zf-apigility#90 (comment)