Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use pagination to find all files #20

Closed
wants to merge 3 commits into from
Closed

Conversation

ruudk
Copy link

@ruudk ruudk commented Oct 5, 2023

Fixes #19

This increases the page size from the default 30 to the maximum of 100.

It then uses octokit.paginate to automatically fetch all pages.

No idea if this works. Let's see what the tests do.

Fixes gkampitakis#19

This increases the page size from the default 30 to the maximum of 100.

It then uses `octokit.paginate` to automatically fetch all pages.
@ruudk
Copy link
Author

ruudk commented Oct 5, 2023

Ok, I just tried this branch out and it solves my problem 🎉

I couldn't get the mocked test to pass, @gkampitakis could you help out? 🙏

@gkampitakis
Copy link
Owner

gkampitakis commented Oct 6, 2023

Updated test

```ts
import { getFiles } from '../src/pr-files';
import { Octokit } from '../src/types';

const sleep = (time: number) =>
  new Promise(resolve => setTimeout(resolve, time));
const execSpy = jest.fn();
const diffCMD = 'git diff origin/master...origin/pr -- src/data.ts';

jest.mock('util', () => {
  const original = jest.requireActual('util');

  return {
    ...original,
    promisify: () => async (command: string) => {
      await sleep(2000);
      execSpy(command);
      return { stdout: 'mock-diff', stderr: '' };
    }
  };
});

const spy = jest.fn();
const mockOctokit = {
  rest: {
    pulls: {
      listFiles: "mock"
    }
  },
  paginate: (mock: string, args: any) => {
    spy(mock)
    spy(args)
    return [
      {
        status: 'added',
        filename: 'src/data.ts',
        additions: 10
      },
      {
        status: 'added',
        filename: 'data.js',
        additions: 4,
        patch: 'mock-patch'
      },
      {
        status: 'removed',
        filename: 'mock-file.spec.ts',
        additions: 4,
        patch: 'mock-patch'
      },
      {
        status: 'unchanged',
        filename: 'more-data.ts',
        additions: 4,
        patch: 'mock-patch'
      },
      {
        status: 'modified',
        filename: 'lib/index.js',
        additions: 0,
        patch: 'mock-patch'
      },
      {
        status: 'modified',
        filename: 'index.ts',
        additions: 2,
        patch: 'mock-patch'
      }
    ]
  },
} as unknown as Octokit;
const owner = 'mock-owner';
const repo = 'mock-repo';
const prNumber = 10;
const base = 'master';
const head = 'pr';

describe('getFiles', () => {
  beforeEach(() => {
    spy.mockClear();
  });

  it('should return correct files', async () => {
    const files = await getFiles({
      octokit: mockOctokit,
      owner,
      repo,
      base,
      head,
      prNumber
    });

    expect(files).toEqual([
      { filename: 'src/data.ts', patch: 'mock-diff' },
      { filename: 'data.js', patch: 'mock-patch' },
      { filename: 'index.ts', patch: 'mock-patch' }
    ]);
    expect(spy).toHaveBeenCalledWith("mock")
    expect(spy).toHaveBeenCalledWith({
      owner,
      repo,
      pull_number: prNumber,
      per_page: 100,
      mediaType: {
        format: 'diff'
      }
    });
    expect(execSpy).toHaveBeenNthCalledWith(1, diffCMD);
  });

  it('should filter out matching files', async () => {
    const files = await getFiles({
      octokit: mockOctokit,
      owner,
      repo,
      prNumber,
      head,
      base,
      ignoreFilesPattern: '.js$'
    });

    expect(files).toEqual([
      { filename: 'src/data.ts', patch: 'mock-diff' },
      { filename: 'index.ts', patch: 'mock-patch' }
    ]);
    expect(spy).toHaveBeenCalledWith({
      owner,
      repo,
      pull_number: prNumber,
      per_page: 100,
      mediaType: {
        format: 'diff'
      }
    });
    expect(execSpy).toHaveBeenNthCalledWith(1, diffCMD);
  });
});

Does the paginate return only the array of objects now?

@ruudk ruudk closed this Mar 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Should use pagination to find more than (default) 30 files on a PR
2 participants