Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NeuralkTalk2: Stuck at "listing all images in directory" after changing DataLoaderRaw.lua file #183

Open
texzone opened this issue Jul 18, 2017 · 0 comments

Comments

@texzone
Copy link

texzone commented Jul 18, 2017

I have installed NeuralTalk2 and, to the best of my knowledge, all dependencies. I am running Ubuntu 14.04 on VMware Workstation 12. I have around 9.7GB of RAM allocated to the VM. When I run free, this is my output:

tex@ubuntu:~/neuraltalk2$ free
             total       used       free     shared    buffers     cached
Mem:       9906788    2806640    7100148      11660      54184    1250844
-/+ buffers/cache:    1501612    8405176
Swap:      1045500          0    1045500

This is the command I'm running (and the output that is displayed afterwards):

tex@ubuntu:~/neuraltalk2$ th eval.lua -model ~/Desktop/model_id1-501-1448236541.t7_cpu.t7 -image_folder ~/catkin_ws/video_images/Best\ Melodic/ -num_images -1 -gpuid -1

tex@ubuntu:~/neuraltalk2$ th eval.lua -model ~/Desktop/model_id1-501-1448236541.t7_cpu.t7 -image_folder ~/catkin_ws/video_images/Best\ Melodic/ -num_images -1 -gpuid -1
DataLoaderRaw loading images from folder: 	/home/tex/catkin_ws/video_images/Best Melodic/	
listing all images in directory /home/tex/catkin_ws/video_images/Best Melodic/	

Its just stuck there. Been stuck there for maybe 30 minutes. There are only 45 jpeg files.

I have tried changin the number of images, deleting images, making only one image available.... I dont know what to do. Its JPEG, do I don't think there should be a problem because its supported, right?

IMPORTANT: I changed the DataLoaderRaw.lua file to this:

--[[
Same as DataLoader but only requires a folder of images.
Does not have an h5 dependency.
Only used at test time.
]]--

local utils = require 'misc.utils'
require 'lfs'
require 'image'

local DataLoaderRaw = torch.class('DataLoaderRaw')

function DataLoaderRaw:__init(opt)
  local coco_json = utils.getopt(opt, 'coco_json', '')

  -- load the json file which contains additional information about the dataset
  print('DataLoaderRaw loading images from folder: ', opt.folder_path)

  self.files = {}
  self.ids = {}
  if string.len(opt.coco_json) > 0 then
    print('reading from ' .. opt.coco_json)
    -- read in filenames from the coco-style json file
    self.coco_annotation = utils.read_json(opt.coco_json)
    for k,v in pairs(self.coco_annotation.images) do
      local fullpath = path.join(opt.folder_path, v.file_name)
      table.insert(self.files, fullpath)
      table.insert(self.ids, v.id)
    end
  else
    -- read in all the filenames from the folder
    print('listing all images in directory ' .. opt.folder_path)
    local function isImage(f)
      local supportedExt = {'.jpg','.JPG','.jpeg','.JPEG','.png','.PNG','.ppm','.PPM'}
      for _,ext in pairs(supportedExt) do
        local _, end_idx =  f:find(ext)
        if end_idx and end_idx == f:len() then
          return true
        end
      end
      return false
    end
    local n = 1
    local m = 0
    local counter = 1
    while counter <= #paths.dir(opt.folder_path) - 2 do -- two directories "." and ".." returned by dir that are not images
      local file_added = false
      for file in paths.files(opt.folder_path, isImage) do
        if tonumber(string.sub(file, 7, 10)) == n then -- insert files into self.files in order
          if tonumber(string.sub(file, 12, 12)) == m then
	    local fullpath = path.join(opt.folder_path, file)
	    table.insert(self.files, fullpath)
	    table.insert(self.ids, tostring(n)) -- just order them sequentially
	    m=m+1
	    counter = counter + 1
	    file_added = true
	    break
	  end
        end
      end
      if file_added == false then
        n = n + 1
        m = 0
      end
    end
    print(self.files)
  end

  self.N = #self.files
  print('DataLoaderRaw found ' .. self.N .. ' images')

  self.iterator = 1
end

function DataLoaderRaw:resetIterator()
  self.iterator = 1
end

--[[
  Returns a batch of data:
  - X (N,3,256,256) containing the images as uint8 ByteTensor
  - info table of length N, containing additional information
  The data is iterated linearly in order
--]]
function DataLoaderRaw:getBatch(opt)
  local batch_size = utils.getopt(opt, 'batch_size', 5) -- how many images get returned at one time (to go through CNN)
  -- pick an index of the datapoint to load next
  local img_batch_raw = torch.ByteTensor(batch_size, 3, 256, 256)
  local max_index = self.N
  local wrapped = false
  local infos = {}
  for i=1,batch_size do
    local ri = self.iterator
    local ri_next = ri + 1 -- increment iterator
    if ri_next > max_index then ri_next = 1; wrapped = true end -- wrap back around
    self.iterator = ri_next

    -- load the image
    local img = image.load(self.files[ri], 3, 'byte')
    img_batch_raw[i] = image.scale(img, 256, 256)

    -- and record associated info as well
    local info_struct = {}
    info_struct.id = self.ids[ri]
    info_struct.file_path = self.files[ri]
    table.insert(infos, info_struct)
  end

  local data = {}
  data.images = img_batch_raw
  data.bounds = {it_pos_now = self.iterator, it_max = self.N, wrapped = wrapped}
  data.infos = infos
  return data
end

Is there an obvious error somewhere? I changed the function DataLoaderRaw:__init(opt) in this script to sort the files loaded by the DataLoader before appending them to self.files(line 52). Without this functionality, captions are generated for images in a random order and the progression of a scene is impossible to determine.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant